changegroup: move source check to top of addchangegroup This is preparation for some refactoring.

error: remove superfluous pass statements

hook: raise a separate exception for when loading a hook fails For easier catching.

clonebundles: advertise clone bundles feature to clients Server operators that have enabled clone bundles probably want clients to use it. This patch introduces a feature that will insert a bundle2 "output" part that advertises the existence of the clone bundles feature to clients that aren't using it. The server uses the "cbattempted" argument to "getbundle" to determine whether a client supports clone bundles and to avoid sending the message to clients that failed the clone bundle for whatever reason.

exchange: advertise if a clone bundle was attempted The client now sends a "cbattempted" boolean flag to the "getbundle" wire protocol command to tell the server whether a clone bundle was attempted. The presence of this flag will enable the server to conditionally emit a bundle2 "output" part advertising the availability of clone bundles to compatible clients that don't have it enabled.

exchange: record that we attempted to fetch a clone bundle This is needed so a subsequent patch can conditionally add a bundle2 part to the "getbundle" wire protocol command depending on whether a clone bundle was attempted.

exchange: provide hint on how to disable clone bundles If a clone bundle persistently fails to apply, users need a way to disable it so they have a hope of the clone working. Change the hint for the abort scenario to advertise the config option to disable clone bundles.

exchange: document filterclonebundleentries

wireproto: properly parse false boolean args (BC) The client represents boolean arguments as '0' and '1'. bool('0') == bool('1') == True, so a simple bool(val) isn't sufficient for converting the argument back to a bool type. Currently, "obsmarkers" is the only boolean argument to getbundle. I /think/ the only place where we currently set the "obsmarkers" argument is during bundle2 pulls. As a result of this bug, the server /might/ be sending obsolete markers bundle2 part(s) to clients that don't request them. That is why I marked this BC. Surprisingly there was no test fall out from this change. I suspect a lapse in test coverage.

bundle2: gracefully skip 'obsmarkers' part if evolution is disabled We would skip the part if it was fully unknown, so we should also skip it if we know we won't be able to apply it. This will allow us to produce bundles with obsolescence markers alongside changegroup while still being able to apply them on any client.

obsstore: make the readonly attribute accessible We want to gracefully handle the read only case in some case (current target: advisory obsmarkers parts in bundle2). So we expose the attribute in a clean way.

update: introduce a 'UpdateAbort' exception The 'postincoming' function used by 'hg pull --update' and 'hg unbundle' is catching 'Abort' exceptions to intercept failed update. This feel a bit too wide to me, so I'm introducing a more precise exception to specify update destination issues.

update: "deprecate" call to 'merge.update' without a destination Now that all internal callers pre-compute and set a destination at a higher level it feels like we can kill this API. This will allow us to simplify this function. However I feel like this is a bit too central and critical to break now. I'm adding a devel warning to let extension make catch this in the next cycle.

shelve: delete shelve statefile on any exception during abort When a user's repository is in an unfinished unshelve state and they choose to abort, at a minimum, the repo should be out of that state. We've found situations where the user could not leave the state unless manually deleting the state file. This fix ensures that no matter what exception may be raised during the abort, the shelved state file will be deleted, the user will be out of the unshelve state and they can get their repository into a workable condition.

highlight: add option to prevent content-only based fallback When Mozilla enabled Pygments on hg.mozilla.org, we got a lot of weirdly colorized files. Upon further investigation, the hightlight extension is first attempting a filename+content based match then falling back to a purely content-driven detection mode in Pygments. Sounds good in theory. Unfortunately, Pygments' content-driven detection establishes no minimum threshold for returning a lexer. Furthermore, the detection code for a number of languages is very liberal. For example, ActionScript 3 will return a confidence of 0.3 (out of 1.0) if the first 1k of the file we pass in matches the regex "\w+\s*:\s*\w"! Python matches on "import ". It's no coincidence that a number of our extension-less files were getting highlighted improperly. This patch adds an option to have the highlighter not fall back to purely content-based detection when filename+content detection failed. This can be enabled to render unlighted text instead of taking the risk that unknown file types are highlighted incorrectly. The old behavior is still the default.

highlight: inline checkfctx() It is only used once. pygmentize() is pretty small. Let's just inline it.

highlight: consolidate duplicate code I'm adding some logic in a future patch and this will make it so I only have to add it once.

rebase: properly abort when destination is public (issue4896) After rebasing a set of changes onto a public changeset and having the first one be skipped, if you try to abort, the operation fails. This fix adds a check to disallow the target rev into the dstates list within the abort function. This list is checked for immutable states before the rest of abort does its thing.

bookmarks: don't deactivate on no-op update (issue4901)

rebase: properly handle chains of markers with missing nodes As obsolescence markers can contains unknown nodes and 'allsuccessors' returns them, we have to protect again that when looking for successors of the rebase set in the destination. Test have been expanded to catch that.

rebase: use a direct reference to repo.changelog Accessing 'repo.changelog' have a small overhead because we double check that the filtering did not changed. As we make multiple use of this into loops, we should avoid doing the lookup/check every time. This also make the code tidier.

mail: drop python 2.5 support

exchange: use pushop.repo instead of repo

rebase: factor out nothing to rebase return code A rebase call that results in nothing to rebase might be considered successful in some contexts. This factors out the return code from places where hg determines that there is nothing to rebase, so an extenion might change this return code to be something that would allow scripts to run without seeing this as an error.

gitweb: visually highlight source lines when hovering over line numbers Due to how the line links now reside outside of the source lines, hovering over line numbers doesn't count as hovering over the appropriate source line. It can be worked around by using a "+" css selector. However, it's necessary to reorder the elements and put <a> before <span> (which is actually quite logical). It works without further css tweaks because <a> is already absolute-positioned and so the order doesn't matter visually.

rebase: added comments Added comments describing the state variable and constants used throughout the rebase extension

monoblue: visually highlight source lines when hovering over line numbers Due to how the line links now reside outside of the source lines, hovering over line numbers doesn't count as hovering over the appropriate source line. It can be worked around by using a "+" css selector. However, it's necessary to reorder the elements and put <a> before <span> (which is actually quite logical). It works without further css tweaks because <a> is already absolute-positioned and so the order doesn't matter visually.

monoblue: make the size of line links bigger to cover line numbers better Due to how the line numbers in monoblue are formed (via css counters), the size of the area with the numbers and the size of the actually clickable links are not tied together well enough. Before this patch, there were noticeable "gaps" between line links - clicking on the bottom part of a visible line number did nothing as opposed to selecting this line. Let's set font-size for everything in pre.sourcelines so that it also affects the links and then add a bit of padding to them so compensate for layout differences. This way the sizes are still not 100% the same, but should be very close.

gitweb: don't drop current revision context on graph page In hgweb, some pages have a context of current revision; e.g. changelog and shortlog show changesets starting from this current revision. However, some gitweb templates were dropping current revision from some urls _to_ /graph page and _on_ that page. This patch fixes it.

util: also catch IndexError This makes life so, so much easier for hgwatchman, which provides a named tuple but throws an IndexError instead of a TypeError.

exewrapper: add comments about PYTHONHOME This has been a repeating source of confusion for users of HackableMercurial. Note that users of HackableMercurial should *not* and are *not* expected to set PYTHONHOME.

exewrapper: report name of failed private DLL in error message Same as in preceding change, but this time it's for the private Python of HackableMercurial (in hg-python directory next to the hg.exe).

exewrapper: report name of failed DLL in error message This uses C string literal concatenation. Example output: $ hg version abort: failed to load Python DLL python27.dll Note that HGPYTHONLIB does not contain the trailing .dll, which works here because the documentation of LoadLibrary [1] says, that if no file name extension is specified in the filename parameter, the default library extension .dll is appended. See [2] for a motivation of this change. [1] https://msdn.microsoft.com/en-us/library/windows/desktop/ms684175(v=vs.85).aspx [2] https://selenic.com/pipermail/mercurial/2015-August/048627.html

merge: abort on file/directory case folding collisions (issue4892) File/directory case folding collisions cannot be represented on case folding systems and have to fail. To detect this and abort early, utilize that for file/directory collisions, a sorted list of case folded manifest names will have the colliding directory right after the file. (This could perhaps be optimized, but this way of doing it also has directory/directory case folding in mind ... which however not is handled yet.)

tests: test of case-folding merge collisions

copies: group bothnew with other sets

copies: rename renamedelete to renamedeleteset for clarity

copies: move _makegetfctx calls into checkcopies

copies: factor out setupctx into _makegetfctx This reduces the scope of mergecopies a bit

copies: avoid reference to c1/c2 in makectx

mq: use cmdutil.revert instead of hg.revert It's the last user.

debugmergestate: add support for printing out driver-resolved files

debugmergestate: add support for printing out merge driver

merge.mergedriver: don't try resolving files marked driver-resolved The driver is expected to take care of these.

merge.mergestate: add support for persisting driver-resolved files A driver-resolved file is a file that's handled specially by the driver. A common use case for this state would be autogenerated files, the generation of which should happen only after all source conflicts are resolved. This is done with an uppercase letter because older versions of Mercurial will not know how to treat such files at all.

merge.mergestate: add support for persisting a custom merge driver A 'merge driver' is a coordinator for the overall merge process. It will be able to control: - tools for individual files, much like the merge-patterns configuration does today - tools that can work across groups of files - the ordering of file resolution - resolution of automatically generated files - adding and removing additional files to and from the dirstate Since it is a critical part of the merge process, it really is part of the merge state. This is a lowercase character (i.e. optional) because ignoring this is fine for older versions of Mercurial -- however, if there are any files that are specially treated by the driver, we should abort. That will happen in upcoming patches. There is a potential security issue with storing the merge driver in the merge state. See the inline comments for more details.

exchange: support sorting URLs by client-side preferences Not all bundles are appropriate for all clients. For example, someone with a slow Internet connection may want to prefer bz2 bundles over gzip bundles because they are smaller and don't take as long to transfer. This is information that a server cannot know on its own. So, we invent a mechanism for "preferring" server-advertised URLs based on their attributes. We could invent a negotiation between client and server where the client sends its preferences and the sorting/filtering is done server-side. However, this feels complex. We can avoid complicating the wire protocol and exposing ourselves to backwards compatible concerns by performing the sorting locally. This patch defines a new config option for expressing preferred attributes in server-advertised bundles. At Mozilla, we leverage this feature so clients in fast data centers prefer uncompressed bundles. (We advertise gzip bundles first because that is a reasonable default.) I consider this an advanced feature. I'm on the fence as to whether it should be documented in `hg help config`.

exchange: extract bundle specification components into own attributes An upcoming patch will enable clients to prefer certain bundles over others. The idea is that we define values of attributes from manifests that are desirable. The BUNDLESPEC attribute is a complex value consisting of multiple parts. Clients may wish to only prefer one of these parts. Having to specify every combination of BUNDLESPEC would be annoying. So, we extract the components of BUNDLESPEC into their own attributes so clients can easily filter on a sub-component.

exchange: support preserving external names when parsing bundle specs This will be needed to make client-side preferences work easier.

clonebundles: filter on SNI requirement Server Name Indication (SNI) is commonly used in CDNs and other hosted environments. Unfortunately, Python <2.7.9 does not support SNI and when these older Python versions attempt to negotiate TLS to an SNI server, they raise an opaque error like "_ssl.c:507: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure." We introduce a manifest attribute to denote the URL requires SNI and have clients without SNI support filter these entries.

clonebundles: filter on bundle specification Not all clients are capable of reading every bundle. Currently, content negotiation to ensure a server sends a client a compatible bundle format is performed at request time. The response bundle is dynamically generated at request time, so this works fine. Clone bundles are statically generated *before* the request. This means that a modern server could produce bundles that a legacy client isn't capable of reading. Without some kind of "type hint" in the clone bundles manifest, a client may attempt to download an incompatible bundle. Furthermore, a client may not realize a bundle is incompatible until it has processed part of the bundle (imagine consuming a 1 GB changegroup bundle2 part only to discover the bundle2 part afterwards is incompatibl). This would waste time and resources. And it isn't very user friendly. Clone bundle manifests thus need to advertise the *exact* format of the hosted bundles so clients may filter out entries that they don't know how to read. This patch introduces that mechanism. We introduce the BUNDLESPEC attribute to declare the "bundle specification" of the entry. Bundle specifications are parsed using exchange.parsebundlespecification, which uses the same strings as the "--type" argument to `hg bundle`. The supported bundle specifications are well defined and backwards compatible. When a client encounters a BUNDLESPEC that is invalid or unsupported, it silently ignores the entry.

clonebundle: support bundle2 exchange.readbundle() can return 2 different types. We weren't handling the bundle2 case. Handle it. At some point we'll likely want a generic API for applying a bundle from a file handle. For now, create another one-off until we figure out what the unified bundle API should look like (addressing this is a can of worms I don't want to open right now).

update: also use 'destupdate' for pull and unbundle Update can also be performed by 'hg pull --update' and 'hg unbundle'. We use the destupdate function in these case too.

destupdate: also include bookmark related logic For the same reason, we move the bookmark related update logic into the 'destupdate' function. This requires to extend the returns of the function to include the bookmark that needs to move (more or less) and the bookmark to activate at the end of the function. See function documentation for details on this returns.

exchange: refactor bundle specification parsing The old code was tailored to `hg bundle` usage and not appropriate for use as a general API, which clone bundles will require. The code has been rewritten to make it more generally suitable. We introduce dedicated error types to represent invalid and unsupported bundle specifications. The reason we need dedicated error types (rather than error.Abort) is because clone bundles will want to catch these exception as part of filtering entries. We don't want to swallow error.Abort on principle.

exchange: move bundle specification parsing from cmdutil Clone bundles require a well-defined string to specify the type of bundle that is listed so clients can filter compatible file types. The `hg bundle` command and cmdutil.parsebundletype() already establish the beginnings of a bundle specification format. As part of formalizing this format specification so it can be used by clone bundles, we move the specification parsing bits verbatim to exchange.py, which is a more suitable place than cmdutil.py. A subsequent patch will refactor this code to make it more appropriate as a general API.

revset: add optional offset argument to limit() predicate It's common for GUI or web frontend to fetch chunk of revisions per batch size. Previously it was possible only if revisions were sorted by revision number. $ hg log -r 'limit({revspec} & :{last_known}, 101)' So this patch introduces a general way to retrieve chunk of revisions after skipping offset revisions. $ hg log -r 'limit({revspec}, 100, {last_count})' This is a dumb implementation. We can optimize it for baseset and spanset later.

revset: port limit() to support keyword arguments The next patch will introduce the third 'offset' argument. This allows us to specify 'offset' without 'n' argument.

revset: eliminate temporary reference to subset in limit() and last()

dirstate: read from pending file under HG_PENDING mode if it exists True/False value of '_pendingmode' means whether 'dirstate.pending' is used to initialize own '_map' and so on. When it is None, neither 'dirstate' nor 'dirstate.pending' is read in yet. This is used to keep consistent view between '_pl()' and '_read()'. Once '_pendingmode' is determined by reading one of 'dirstate' or 'dirstate.pending' in, '_pendingmode' is kept even if 'invalidate()' is invoked. This should be reasonable, because: - effective 'invalidate()' invocation should occur only in wlock scope, and - wlock can't be gotten under HG_PENDING mode '_trypending()' is defined as a normal function to factor similar code path (in bookmarks and phases) out in the future easily.

dirstate: make writing in-memory changes aware of transaction activity This patch delays writing in-memory changes out, if transaction is running. '_getfsnow()' is defined as a function, to hook it easily for ambiguous timestamp tests (see also fakedirstatewritetime.py) 'if tr:' code path in this patch is still disabled at this revision, because there is no client invoking 'dirstate.write()' with repo object. BTW, this patch changes 'dirstate.invalidate()' semantics around 'dirstate.write()' in a transaction scope: before: with repo.transaction(): dirstate.CHANGE('A') dirstate.write() # change for A is written out here dirstate.CHANGE('B') dirstate.invalidate() # discards only change for B after: with repo.transaction(): dirstate.CHANGE('A') dirstate.write() # change for A is still kept in memory dirstate.CHANGE('B') dirstate.invalidate() # discards changes for A and B Fortunately, there is no code path expecting the former, at least, in Mercurial itself, because 'dirstateguard' was introduced to remove such 'dirstate.invalidate()'.

dirstate: make functions for backup aware of transaction activity Some comments in this patch assume that subsequent patch changes 'dirstate.write()' like as below: def write(self, repo): if not self._dirty: return tr = repo.currenttransaction() if tr: tr.addfilegenerator('dirstate', (self._filename,), self._writedirstate, location='plain') return # omit actual writing out st = self._opener('dirstate', "w", atomictemp=True) self._writedirstate(st) This patch makes '_savebackup()' write in-memory changes out, and it causes clearing 'self._dirty'. If dirstate isn't changed after '_savebackup()', subsequent 'dirstate.write()' never invokes 'tr.addfilegenerator()' because 'not self._dirty' is true. Then, 'tr.writepending()' unintentionally returns False, if there is no other (e.g. changelog) changes pending, even though dirstate changes are already written out at '_savebackup()'. To avoid such situation, this patch makes '_savebackup()' explicitly invoke 'tr.addfilegenerator()', if transaction is running. '_savebackup()' should get awareness of transaction before 'write()', because the former depends on the behavior of the latter before this patch.

dirstate: move code paths for backup from dirstateguard to dirstate This can centralize the logic to write in-memory changes out correctly according to transaction activity into dirstate. Passing 'repo' object to newly added functions is needed to examine current transaction activity in subsequent patches, because 'dirstate' itself doesn't have direct reference to it.

localrepo: restore dirstate to one before rollbacking if not parent-gone 'localrepository.rollback()' explicilty restores dirstate, only if at least one of current parents of the working directory is removed at rollbacking (a.k.a "parent-gone"). After DirstateTransactionPlan, 'dirstate.write()' will cause marking '.hg/dirstate' as a file to be restored at rollbacking. https://mercurial.selenic.com/wiki/DirstateTransactionPlan Then, 'transaction.rollback()' restores '.hg/dirstate' regardless of parents of the working directory at that time, and this causes unexpected dirstate changes if not "parent-gone" (e.g. "hg update" to another branch after "hg commit" or so, then "hg rollback"). To avoid such situation, this patch restores dirstate to one before rollbacking if not "parent-gone". before: b1. restore dirstate explicitly, if "parent-gone" after: a1. save dirstate before actual rollbacking via dirstateguard a2. restore dirstate via 'transaction.rollback()' a3. if "parent-gone" - discard backup (a1) - restore dirstate from 'undo.dirstate' a4. otherwise, restore dirstate from backup (a1) Even though restoring dirstate at (a3) after (a2) seems redundant, this patch keeps this existing code path, because: - it isn't ensured that 'dirstate.write()' was invoked at least once while transaction running If not, '.hg/dirstate' isn't restored at (a2). In addition to it, rude 3rd party extension invoking 'dirstate.write()' without 'repo' while transaction running (see subsequent patches for detail) may break consistency of a file backup-ed by transaction. - this patch mainly focuses on changes for DirstateTransactionPlan Restoring dirstate at (a3) itself should be cheaper enough than rollbacking itself. Redundancy will be removed in next step. Newly added test is almost meaningless at this point. It will be used to detect regression while implementing delayed dirstate write out.

parsers: make pack_dirstate take now in integer for consistency On recent OS, 'stat.st_mtime' has a double precision floating point value to represent nano seconds, but it is not wide enough for actual file timestamp: nowadays, only 52 - 32 = 20 bit width is available for decimal places in sec. Therefore, casting it to 'int' may cause unexpected result. See also changeset 13272104bb07 fixing issue4836 for detail. For example, changed file A may be treated as "clean" unexpectedly in steps below. "rounded now" is the value gotten by rounding via 'int(st.st_mtime)' or so. ---------------------+--------------------+------------------------ "now" | | timestamp of A (time_t) float rounded time_t| action | FS dirstate ------ ------- ------+--------------------+-------- --------------- N+.nnn N N | | --- --- | update file A | N | dirstate.normal(A) | N N+.999 N+1 N | | | dirstate.write() | N (*1) | : | | change file A | N | : | N+1.00 N+1 N+1 | | | "hg status" (*2) | N N ------ ------- ------+--------------------+-------- --------------- Timestamp N of A in dirstate isn't dropped at (*1), because "rounded now" is N+1 at that time, even if 'st_mtime' in 'time_t' is still N. Then, file A is unexpectedly treated as "clean" at (*2) in this case. For consistent handling of 'stat.st_mtime', this patch makes 'pack_dirstate()' take 'now' argument not in floating point but in integer. This patch makes 'PyArg_ParseTuple()' in 'pack_dirstate()' use format 'i' (= checking type mismatch or overflow), even though it is ensured that 'now' is in the range of 32bit signed integer by masking with '_rangemask' (= 0x7fffffff) on caller side. It should be cheaper enough than packing itself, and useful to detect that legacy code invokes 'pack_dirstate()' with 'now' in floating point value.

destupdate: include the 'check' logic After moving logic from 'merge.update' into 'destutil.destupdate', we are now moving logic from 'command.update' in 'destutil.destupdate'. This will make the function actually useful in predicting (and altering) the update behavior.

destupdate: move the check related to the "clean" logic in the function We want this function to exactly predict the behavior for update. Moreover, we would like to remove all high level behavior logic out of the merge module so this is a step forward. Now that the 'destupdate' function both compute and validate the destination, we can directly use it at the command level, ensuring that the 'hg update' command never call 'merge.update' without a defined destination. This is a first (but significant) step toward having 'merge.update' always feed with a properly validated destination and free of high level logic.

largefiles: better handling of merge of largefiles that not are available Before, when merging revisions with missing largefiles, the missing largefiles would be fetched as a part of the merge. If that failed (for example because the main repository temporarily was unavailable), the largefile would be left missing. However, the next commit would abort and (seemed to) fail when markcommitted tried to mark the standin file as normal and thus had to hash the largefile that didn't exist. (Actually, the commit would succeed but the largefile update that follows right after the commit transaction would abort - quite confusing.) To fix that, make sure that synclfdirstate only marks files as normal if they actually exist.

patchbomb: check that targets exist at the publicurl Advertising that the patch are available to be pulled requires that to be true. So we check revision availability on the remote before sending any email.

windows: read all global config files, not just the first (issue4491) (BC) On windows, hgrc.d/*.rc would not be read if mercurial.ini was found. That was far from obvious from the documentation and different from the behavior on posix systems. As a consequence of this, TortoiseHg cacert configuration placed in hgrc.d would not be read if an old global mercurial.ini still existed. "hg config -g" could also crash when no global configuration files could be found. Instead, make windows behave like posix and read all global configuration files. The documentation was in a way right that individual config settings in the global Mercurial.ini would override settings from for example .hgrc.d\*.rc, but only because the .d files not would be read at all if a Mercurial.ini was found. The ordering in the documentation is thus changed to match the code.

strip: factor out revset calculation for strip -B This will allow reusing it in evolve and overriding it in other extensions.

clonebundles: support for seeding clones from pre-generated bundles Cloning can be an expensive operation for servers because the server generates a bundle from existing repository data at request time. For a large repository like mozilla-central, this consumes 4+ minutes of CPU time on the server. It also results in significant network utilization. Multiplied by hundreds or even thousands of clients and the ensuing load can result in difficulties scaling the Mercurial server. Despite generation of bundles being deterministic until the next changeset is added, the generation of bundles to service a clone request is not cached. Each clone thus performs redundant work. This is wasteful. This patch introduces the "clonebundles" extension and related client-side functionality to help alleviate this deficiency. The client-side feature is behind an experimental flag and is not enabled by default. It works as follows: 1) Server operator generates a bundle and makes it available on a server (likely HTTP). 2) Server operator defines the URL of a bundle file in a .hg/clonebundles.manifest file. 3) Client `hg clone`ing sees the server is advertising bundle URLs. 4) Client fetches and applies the advertised bundle. 5) Client performs equivalent of `hg pull` to fetch changes made since the bundle was created. Essentially, the server performs the expensive work of generating a bundle once and all subsequent clones fetch a static file from somewhere. Scaling static file serving is a much more manageable problem than scaling a Python application like Mercurial. Assuming your repository grows less than 1% per day, the end result is 99+% of CPU and network load from clones is eliminated, allowing Mercurial servers to scale more easily. Serving static files also means data can be transferred to clients as fast as they can consume it, rather than as fast as servers can generate it. This makes clones faster. Mozilla has implemented similar functionality of this patch on hg.mozilla.org using a custom extension. We are hosting bundle files in Amazon S3 and CloudFront (a CDN) and have successfully offloaded >1 TB/day in data transfer from hg.mozilla.org, freeing up significant bandwidth and CPU resources. The positive impact has been stellar and I believe it has proved its value to be included in Mercurial core. I feel it is important for the client-side support to be enabled in core by default because it means that clients will get faster, more reliable clones and will enable server operators to reduce load without requiring any client-side configuration changes (assuming clients are up to date, of course). The scope of this feature is narrowly and specifically tailored to cloning, despite "serve pulls from pre-generated bundles" being a valid and useful feature. I would eventually like for Mercurial servers to support transferring *all* repository data via statically hosted files. You could imagine a server that siphons all pushed data to bundle files and instructs clients to apply a stream of bundles to reconstruct all repository data. This feature, while useful and powerful, is significantly more work to implement because it requires the server component have awareness of discovery and a mapping of which changesets are in which files. Full, clone bundles, by contrast, are much simpler. The wire protocol command is named "clonebundles" instead of something more generic like "staticbundles" to leave the door open for a new, more powerful and more generic server-side component with minimal backwards compatibility implications. The name "bundleclone" is used by Mozilla's extension and would cause problems since there are subtle differences in Mozilla's extension. Mozilla's experience with this idea has taught us that some form of "content negotiation" is required. Not all clients will support all bundle formats or even URLs (advanced TLS requirements, etc). To ensure the highest uptake possible, a server needs to advertise multiple versions of bundles and clients need to be able to choose the most appropriate from that list one. The "attributes" in each server-advertised entry facilitate this filtering and sorting. Their use will become apparent in subsequent patches. Initial inspiration and credit for the idea of cloning from static files belongs to Augie Fackler and his "lookaside clone" extension proof of concept.

sslutil: expose attribute indicating whether SNI is supported This will be used so clone bundles can advertise whether URLs require SNI. This will be explained more in a subsequent patch.

resolve: perform all premerges before performing any file merges (BC) Just like the BC to merge before it, this allows for a maximally consistent state before providing any prompts to the user.

test-resolve.t: add some tests for .orig file contents An upcoming patch will touch some code around this area, and I couldn't find any tests related to this.

test-resolve.t: add some output to show order of operations This basically shows the behavior of resolve with multiple files. An upcoming behavior change will cause this output to also change.

merge.mergestate: perform all premerges before any merges (BC) We perform all that we can non-interactively before prompting the user for input via their merge tool. This allows for a maximally consistent state when the user is first prompted. The test output changes indicate the actual behavior change happening.

merge: introduce a preresolve function The section of code that writes out the version of the file cached in the merge state should only be run at preresolve time. This is so that if the premerge keeps around conflict markers, those don't get overwritten before the main merge.

merge.mergestate._resolve: also return completed status We'll need this for a new 'preresolve' function we're adding.

merge.mergestate: add a wrapper around resolve The resolve function will be broken up into separate pre-resolve and resolve steps.

simplemerge: move conflict warning message to filemerge The current output for a failed merge with conflict markers looks something like: merging foo warning: conflicts during merge. merging foo incomplete! (edit conflicts, then use 'hg resolve --mark') merging bar warning: conflicts during merge. merging bar incomplete! (edit conflicts, then use 'hg resolve --mark') We're going to change the way merges are done to perform all premerges before all merges, so that the output above would look like: merging foo merging bar warning: conflicts during merge. merging foo incomplete! (edit conflicts, then use 'hg resolve --mark') warning: conflicts during merge. merging bar incomplete! (edit conflicts, then use 'hg resolve --mark') The 'warning: conflicts during merge' line has no context, so is pretty confusing. This patch will change the future output to: merging foo merging bar warning: conflicts while merging foo! (edit, then use 'hg resolve --mark') warning: conflicts while merging bar! (edit, then use 'hg resolve --mark') The hint on how to resolve the conflicts makes this a bit unwieldy, but solving that is tricky because we already hint that people run 'hg resolve' to retry unresolved merges. The 'hg resolve --mark' mostly applies to conflict marker based resolution.

filemerge: clean up some dead code We now exit early if we do a premerge, so extra checks are no longer necessary.

run-tests: add b-prefix on two strings to fix python3 support

filemerge: break overall filemerge into separate premerge and merge steps This means that in ms.resolve we must call merge after calling premerge. This doesn't yet mean that all premerges happen before any merges -- however, this does get us closer to our goal. The output differences are because we recompute the merge tool. The only user-visible difference caused by this patch is that if the tool is missing we'll print the warning twice. Not a huge deal, though.

filemerge: only copy to backup during premerge step The premerge might leave the original file in an unclean state. Therefore it's important to only copy the file in the beginning.

filemerge: only print out "merging f" output at premerge step We're soon going to call this function twice, once for premerge and once for merge. This makes sure the "merging" output only gets printed during the premerge step.

filemerge: deindent the parts of filemerge outside the try block It is no longer necessary to indent these parts.

filemerge: introduce a premerge flag and function This flag will let us get to our overall goal of performing all premerges before any merges.

filemerge: also return whether the merge is complete In future patches, we'll pause merges after the premerge step. After the premerge step we'll return complete = False.

filemerge: add a wrapper around the filemerge function We'll introduce a separate premerge function that calls the same code.

context: don't hex encode all unknown 20 char revision specs (issue4890) d3908c911d5e introduced nice hexified display of missing nodes. It did however also make missing 20 character revision specifications be shown as hex - very confusing. Users are often wrong and somehow specify revisions that don't exist. Nodes will however rarely be missing ... and they will only look like a user provided revision specification and be all ascii in 1 of 4*10**9. With this change, missing revisions will only be hexified if they really look like binary nodes. This change will thus improve the error reporting UI in the common case and only very rarely make it confusing in the opposite direction of how it was before.

discovery: put trivial branch first Having the simple and tiny branch of the conditional first help readability. The "else" that appears after a screen of code is harder to relate to a conditional.

shelve: rename 'publicancestors' to something accurate (issue4737) That function is actually not returning public ancestors at all. This is pointed by the second line of the docstring... The bundling behavior was made correct in a5141977198d but with confusion remaining regarding what each function was doing. This close issue4737, because this highlight that shelve is actually -not- bundling too much data (this was actually properly tested).

makefile: add wheel build target

setup: import setup from setuptools if FORCE_SETUPTOOLS is set This should allow easier experimentation with using setuptools in mercurial's build automation, without breaking anything that currently depends on distutils behavior

hgweb: remove obsolete -webkit-border-radius property

monoblue: add a link to the latest file revision For reference, this was added to paper/coal in bb00a159e594 and to gitweb in b3b57ecbda50.

discovery: reference relevant bug in the faulty code We extend the comment about this code flaw with more code flaw.

discovery: fix a typo in a comment The idea here is that the code is imperfect, not that it is impossible to get something behaving properly.

getsubset: get the unpacker version from the bundler The current setup requires to pass both a packer and, optionally, the version of the unpacker. This is confusing and error prone as the two value cannot mismatch. Instead, we simply grab the version from the packer. This fixes a bug where requesting a cg2 from 'hg bundle' were reported as changegroup 1. I should have caught that in the initial changeset but I missed it somehow.

test-convert-cvs: add sleep so cvs notices changes This change makes the test pass on gcc112.fsffrance.org.

cvsps: fix computation of parent revisions when log caching is on cvsps computes the parent revisions of log entries by walking the cvs log sorted by (rcs, revision) and by iteratively maintaining a 'versions' dictionary which maps a (rcs, branch) pair onto the last revision seen for that pair. When log caching is on and a log cache exists, cvsps fails to set the parent revisions of new log entries because it does not iterate over the log cache in the parents computation. A complication is that a file rcs can change (move to/from the attic), with respect to its value in the log cache, if the file is removed/added back. This patch adds an iteration over the log cache to update the rcs of cached log entries, if changed, and to properly populate the 'versions' dictionary.

dirstate: batch calls to statfiles (issue4878) This makes it more interruptible.

parsers: fix infinite loop or out-of-bound read in fm1readmarkers (issue4888) The issue4888 was caused by 0-length obsolete marker. If msize is zero, fm1readmarkers() never ends. This patch adds several bound checks to fm1readmarker(). Therefore, 0-length and invalid-size marker should be rejected.

parsers: read sizes of metadata pair of obsolete marker at once This will make it easy to implement bound checking. Currently fm1readmarker() has no protection for corrupted obsstore and can cause infinite loop or out-of-bound reads.

filemerge: clean up temp files in a finally block This isn't really a big deal because the temp files are created in $TMPDIR, but it makes some upcoming work simpler.

check-code: detect and ban 'util.Abort' We have seen the light, please use the new way.

error: get Abort from 'error' instead of 'util' The home of 'Abort' is 'error' not 'util' however, a lot of code seems to be confused about that and gives all the credit to 'util' instead of the hardworking 'error'. In a spirit of equity, we break the cycle of injustice and give back to 'error' the respect it deserves. And screw that 'util' poser. For great justice.

eol: rename 'error' to 'haserror' The variable 'error' conflict with the module name that we would like to import and use in a coming changeset.

discovery: rename 'error' to 'errormsg' The variable 'error' conflict with the module name that we would like to import and use in a coming changeset.

histedit: delete histedit statefile on any exception during abort When an user aborts a histedit, many things could go wrong. At a minimum, after a histedit abort failure, their repository should be out of that state. We've found situations where the user could not exit the histedit state without manually deleting the histedit state file. This patch ensures that if any exception happens during an abort, the histedit statefile will be deleted so that users are out of the histedit state and can at least manually get the repo back to a workable condition.

histedit: check presence of statefile before deleting it When the histeditstate class instance has it's clear() method called, there is nothing to check to see if the state file exists before deleting it. It may not exist, which would create an exception. This patch allows clear to be called at any time. This will be needed for the following patch.

histedit: add inprogress method to state class If a histedit is progress, the 'histedit-state' file should exist. The patch implements a convenience function to do check if a histedit is in progress. This method will be use in next patch in the series.

commands: use dirstateguard instead of begin/end-parentchange for backout Before this patch, "hg backout" uses 'begin'/'end'-'parentchange()' of 'dirstate' class to avoid writing incomplete dirstate changes out at failure. But this framework doesn't work as expected, if 'dirstate.write()' is invoked between them. In fact, in-memory dirstate changes may be written out at 'repo.status()' implied by 'merge.update()', even before this patch. To restore dirstate as expected at failure of "hg backout", this patch uses 'dirstateguard' instead of 'begin'/'end'-'parentchange()'.

commands: make "hg import" use dirstateguard only for --no-commit Previous patch made dirstate changes in a transaction scope "all or nothing". Therefore, 'dirstateguard' is meaningless, if its scope is as same as one of the related transaction. Before this patch, "hg import" uses 'dirstateguard' always, but transaction is also started if '--no-commit' isn't specified. To avoid redundancy, this patch makes "hg import" use dirstateguard only if transaction isn't started (= '--no-commit' is specified). In this patch, 'if dsguard' can be examined safely, because 'dsguard' is initialized (with None) before outermost 'try'.

cmdutil: stop tryimportone from using dirstateguard (BC) There is no user of 'cmdutil.tryimportone()' other than 'commands.import_()', which can restore dirstate at failure of applying patches by transaction or dirstateguard. Therefore, it is reasonable to stop 'tryimportone()' from using redundant 'dirstateguard', even though it changes behavior of 'tryimportone()'. After this patch, 3rd party extensions should use 'dirstateguard' or so explicitly, if they want to restore dirstate at failure of importing a patch.

dirstate: remove meaningless dirstateguard Previous patch made dirstate changes in a transaction scope "all or nothing". Therefore, 'dirstateguard' is meaningless, if its scope is as same as one of the related transaction. This patch removes such meaningless 'dirstateguard' usage.

localrepo: execute appropriate actions for dirstate at releasing transaction Before this patch, in-memory dirstate changes are still kept over a transaction scope boundary regardless of the result of it. For "all or nothing" policy of the transaction, in-memory dirstate changes should be: - written out at successful closing a transaction, because subsequent 'dirstate.invalidate()' can lose them - discarded at failure of a transaction, because outer 'wlock.release()' or so may write them out To discard all changes in a transaction completely, this patch also restores '.hg/dirstate' by '.hg/journal.dirstate' at failure, because 'transaction' itself does nothing for files related to '.hg/journal.*' in such case (therefore, renaming in this patch is safe enough). This is a part of preparations for "transactional dirstate". See also the wiki page below for detail about it. https://mercurial.selenic.com/wiki/DirstateTransactionPlan This patch also removes redundant 'dirstate.invalidate()' just before aborting a transaction for shelve/unshelve.

transaction: add releasefn to notify the end of a transaction scope 'releasefn' is used by subsequent patch, to do appropriate action according to the result of it at the end of a transaction scope. To ensure that 'releasefn' is invoked only once, this patch invokes it after assignment 'self.journal = None', because such assignment prevents from invoked 'transaction._abort()' again via '__del__()'. def __del__(self): if self.journal: self._abort()

filemerge: move post-merge checks into a separate function This makes the overall filemerge function easier to follow, and makes upcoming work simpler.