test-merge5: remove obsolete '-y' and its motivating comment The '-y' in 'hg update -y' was once needed to answer questions about modify/delete conflicts. That is no longer needed, so remove the '-y' and the comment justifying its use.

test-resolve.t: use redirection to /dev/null instead of grep -q

resolve: run happily after conflict-free merge

test-resolve: add tests for re-merge

test-resolve: add more tests for in conflict-free states We already have a test for 'hg resolve -m' when there is no merge in progress. Add one for 'hg resolve --all' as well. Also add tests for both --all and -m when there is a merge without conflicts in progress. They should both be successful, just as if there had been conflicts that had been marked resolved. However, that is currently broken, so mark the tests broken for now. The behavior will be fixed in a later patch.

test-resolve: add test resolving one of two files The tests for resolve are missing a lot of cases. Let's start by adding another file to the test repo, so we can test resolving one of two files.

test-resolve: clarify test descriptions and consistently use "should"

revset-phases: prefetch attributes in phasesrelated revsets Pre-fetching attributes gives a significant performance boost. Such is Python. draft() 0) wall 0.011661 comb 0.010000 user 0.010000 sys 0.000000 (best of 205) 1) wall 0.009804 comb 0.000000 user 0.000000 sys 0.000000 (best of 231) draft() - ::bookmark() 0) wall 0.014173 comb 0.010000 user 0.010000 sys 0.000000 (best of 177) 1) wall 0.012966 comb 0.010000 user 0.010000 sys 0.000000 (best of 182)

revset-phases: do not cache phase-related filters The phase retrieval is fast enough to not require caching the result of the functions. draft() 0) wall 0.017209 comb 0.020000 user 0.020000 sys 0.000000 (best of 149) 1) wall 0.011654 comb 0.010000 user 0.010000 sys 0.000000 (best of 186) public() 0) wall 0.018687 comb 0.010000 user 0.010000 sys 0.000000 (best of 128) 1) wall 0.013290 comb 0.010000 user 0.010000 sys 0.000000 (best of 181) secret() 0) wall 0.017464 comb 0.020000 user 0.020000 sys 0.000000 (best of 127) 1) wall 0.011499 comb 0.000000 user 0.000000 sys 0.000000 (best of 196) draft() - ::bookmark() 0) wall 0.020099 comb 0.020000 user 0.020000 sys 0.000000 (best of 127) 1) wall 0.014399 comb 0.020000 user 0.020000 sys 0.000000 (best of 169)

changectx: issue a FilteredRepoLookupError when applicable We capture FilteredxxxError and issue a FilteredRepoLookupError instead with a sightly different messsge. The message will likely get more improvement in the future. error: filtered revision '4'

repoview: add a FilteredRepoLookupError This exception is a more precise RepoLookupError that will allow us to issue a special message when we end up accessing a filtered revision.

repoview: add a FilteredLookupError class This exception is a more precise LookupError that will allow us to issue a special message when we end up accessing a filtered revision.

repoview: add a FilteredIndexError class This exception is a more precise IndexError that will allow us to issue a special message when we end up accessing a filtered revision.

changectx: move `IndexError` handling in the top level try except This one can be handled by the top level catching.

changectx: wrap the `changeid` processing in a try/except We are going to introduce more precise exception classes for filtered nodes. So we will have to upgrade them to the `RepoLookupError` level here. We wrap the whole thing into a try/except to ease this future catching. Some of the current exception catching will be moved in this one. But the current changeset focuses on code movement only.

bundle2: detect and disallow a negative chunk size We have no usage planned for 2/3 of them and the support for the planned usecase is not here yet. So we raise a BundleValueError when encountered

bundle2: add an UnsupportedPartError We need the BundleValueError for format errors not related to part support. So we add a specific class for part-support errors.

bundle2: change header size and make them signed (new format) We are changing all integers that denote the size of a chunk to read to int32. There are two main motivations for that. First, we change everything to the same width (32 bits) to make it possible for a reasonably agnostic actor to forward a bundle2 without any extra processing. With this change, this could be achieved by just reading int32s and forwarding chunks of the size read. A bit a smartness would be logic to detect the end of stream but nothing too complicated. Second, we need some capacity to transmit special information during the bundle processing. For example we would like to be able to raise an exception while a part is being read if this exception happend while this part was generated. Having signed integer let us use negative numbers to trigger special events during the parsing of the bundle. The format is renamed for B2X to B2Y because this breaks binary compatibility. The B2X format support is dropped. It was experimental to allow this kind of things. All elements not directly related to the binary format remain flagged "b2x" because they are still compatible.

bundle2: extract processing of part into its own function This is code movement only. This will be useful to have it separated for reuse purposes. We plan to introduce a new feature to the bundle format that allow inserting a part in the middle of another part payload. This will be useful to transmit a exception raised during a part generation.

bundle2: add a test for exceptions raised during the generation process We would like exceptions raised during the generation process to be gracefully handled on the receiver side. We add a test for it. It shows that we are not doing it yet.

dagwalker: drop a useless intermediate variable The variable used to contain revs.set() but as the `.set()` has been removed it is now useless.

revset-node: speedup by a few hundred fold Instead of checking all elements of the subset against a single rev, just check if this rev is in the subset. The old way was inherited from when the subset was a list. Non surprise, this provide massive speedup. id("d82e2223f132") before) wall 0.008205 comb 0.000000 user 0.000000 sys 0.000000 (best of 302) after) wall 0.000069 comb 0.000000 user 0.000000 sys 0.000000 (best of 34518) revset #1: public() and id("d82e2223f132") before) wall 0.019763 comb 0.020000 user 0.020000 sys 0.000000 (best of 124) after) wall 0.000101 comb 0.000000 user 0.000000 sys 0.000000 (best of 20130)

revset-only: use `subset &` instead of filtering The & version is more likely to be optimised. only(.) before) wall 0.003216 comb 0.000000 user 0.000000 sys 0.000000 (best of 768) after) wall 0.001086 comb 0.000000 user 0.000000 sys 0.000000 (best of 2231) only(default, stable) before) wall 0.018469 comb 0.020000 user 0.020000 sys 0.000000 (best of 138) after) wall 0.015888 comb 0.010000 user 0.010000 sys 0.000000 (best of 156)

revset-_ancestor: use & instead of filter The & operation is more likely optimised. ::10 before) wall 0.028189 comb 0.030000 user 0.030000 sys 0.000000 (best of 100) after) wall 0.001050 comb 0.000000 user 0.000000 sys 0.000000 (best of 2326) ::tip before) wall 0.081132 comb 0.080000 user 0.080000 sys 0.000000 (best of 100) after) wall 0.055418 comb 0.050000 user 0.050000 sys 0.000000 (best of 100)

obsstore: record data as floating point in fm0 format For python struct module, "d" is double. But for python string formating, "d" is integer. We want to preserve the floating point nature of the data, so we store it in the metadata as floating point. We use "%r" to make sure we get as many significant digitis as necessary to restore the float to the exact same value on the other side. The fm1 is transmitting the information as float. The lack of this made fm1-stored markers not survive a round-trip to fm0 leading to duplicated markers (or two markers very alike).

bundle2: add a comment about addchangegroup source and url

pull: call the `b2x-transactionclose` hook when closing the transaction We need a wider set of hooks to process all the changes that happened during the pull transaction. We reuse the experimental `b2x-transactionclose` hook set from server's unbundle for consistency. This hook is experimental and will not remains as-is forever, but this will open the door for experimentation in 3.2.

revset-only: use __nonzero__ to check if a revset is empty For some smartsets, computing length is more expensive than checking if the set is empty.

_spanset: drop __getitem__ implementation It is expensive and not part of the official smartset API.

filteredset: drop __getitem__ implementation It is expensive and not part of the official smartset API.

generatorset: implement __len__ It was the only smartset class without a `__len__` implementation.

revset: make __len__ part of the offical API It is common for code to ask for the length of a revset. In fact, all but generatorset already implement it.

cmdserver: add service that listens on unix domain socket and forks process Typical use case of 'unix' mode is a background hg daemon. $ hg serve --cmdserver unix --cwd / -a /tmp/hg-`id -u`.sock Unlike 'pipe' mode in which parent process keeps stdio channel, 'unix' server can be detached. So clients can freely connect and disconnect from server, saving Python start-up time. It might be better to write "--cmdserver socket -a unix:/sockpath" instead of "--cmdserver unix -a /sockpath" in case hgweb gets the ability to listen on unix domain socket.

test-commandserver: add connector for unix domain socket server The next patch will introduce --cmdserver unix.

test-commandserver: allow check() to make connection in different way The next patch will add connector for 'unix' mode server.

test-commandserver: remove unused repopath argument from check() Instead of repopath, check() will receive connect() function as argument. It will allow to connect to server of different mode.

cmdserver: make server streams switchable In 'unix' mode, server instance will be created per connection, and fin/fout are set to socket files.

cmdserver: switch service objects by mode server class will be changed to accept fin/fout pair instead of mode string so that it can interact with socket files.

cmdserver: wrap 'pipe' mode server by service object This is the stub for new mode that will listen for connections on unix domain socket. Though --daemon option is not banned in 'pipe' mode, it is useless because the detached 'pipe' mode server exits immediately due to null stdin. Should it abort if --daemon is specified with --cmdserver pipe or --stdio?

histedit: miscellaneous style cleanups Fix some mis-wrapped lines, re-wrap some lines to be more compact, remove superfluous parens.

histedit: update docstring on histeditstate.read() The docstring showed the previous class-less heritage of read(). Now that it's classy, update the docstring.

histedit: remove now-superfluous repo argument from processreplacement Spotted by Olle on the mailing list during review.

histedit: move locks into state Allow action functions to control the locks. This is necessary for an implementation of x/exec or similar.

histedit: read state from histeditstate Read the state in histeditstate. This allows us to correctly update internal variables when necessary without having to recreate a new state. When we read a state in _histedit state while we will already have state passed from histedit(), we can read the state in place and don't have to merge two histeditstates.

histedit: pass state to action functions

histedit: pass state to processreplacement

histedit: pass state to boostrapcontinue Pass the state to boostrapcontinue and remove the unecessary passing of repo.

histedit: let the state expose a context but serialize correctly to nodes The histedit code often expects a context. However histedit hands around the tuple for the serialization and therefore hand over a parentctxnode. This leads to code having to return a context based on the parentctxnode. We let the state only return a context but correctly serialize and deserialze to a node.

histedit: add clear method to remove state Encapsulate the unlinking histedit-state and use the vfs layer instead of os.unlink.

histedit: use state object where necessary We are using the properties from the state object where necessary and ensure that the state is set correctly.

histedit: add histedit state class Add an histeditstate class that is intended to hold the current state. This allows us encapsulate the state and avoids passing around a tuple which is based on the serialization format. In particular this will give actions more control over the state and allow external sources to have more control of histedits behavior, e.g. an external implementation of x/exec.

import-checker: check modules for pure Python build correctly Before this patch, "import-checker.py" just replaces "/" in specified filenames by ".". This makes modules for pure Python build belong to "mercurial.pure" package, and prevents "import-checker.py" from correctly checking about cyclic dependency in them. This patch discards "pure" component from fully qualified name of such modules. To avoid discarding "pure" from the module name of standard libraries unexpectedly, this patch allows "dotted_name_of_path" to discard "pure" only from Mercurial specific modules, which are specified via command line arguments.

import-checker: treat "from mercurial import XXXX" style correctly Before this patch, "import-checker.py" assumes that the name of Mercurial module recognized by "imported_modules" doesn't have package part: for example, "util". This is reason why "import-checker.py" always builds fully qualified module name up relatively, if the given module doesn't belong to standard Python library. But in fact, modules imported in "from mercurial import XXXX" style already have fully qualified name: for example, "mercurial.util" module imported by "mercurial.parsers" is treated as "mercurial.mercurial.util" because of building module name up relatively. This prevents "import-checker.py" from correctly checking about cyclic dependency in them. This patch avoids building module name up relatively, also if module name starts with "mercurial.", to treat modules imported in "from mercurial import XXXX" style correctly.

encoding: avoid cyclic dependency around "parsers" in pure Python build 80f2b63dd83a brought "asciilower" and "import parsers" into "encoding.py". This works fine with "parsers" module in C implementation, but doesn't with one in pure Python implementation, because the latter causes cyclic dependency below and aborting execution: util => i18n => encoding => parsers => util This patch delays importing "parsers" module until it is really needed, to avoid cyclic dependency around "parsers" in pure Python build.

pull: add source information to the transaction The source information can, should be applied once when opening the transaction for the pull. This will lets element processed within a bundle2 be aware of them and open the door to running a set of hooks when closing this pull transaction. This is similar to what is done in server's unbundle call.

changegroup: store source and url in the `hookargs` dict We store the source and url of the current data into `transaction.hookargs` this let us inherit it from upper layers that may have created a much wider transaction. We have to modify bundle2 at the same time to register the source and url in the transaction. We have to do it in the same patch otherwise, the `addchangegroup` call would fill these values and the hook calling will crash because of the duplicated 'source' and 'url' arguments passed to the hook call.

prechangegroup: use hook argument from the transaction There can be useful data in there (eg: bundle2 related one)

addchangegroup: call `prechangegroup` hook after transaction retrieval We want to reused some possible information stored in the transaction `hookargs` dict that may be stored by something handling the transaction at an upper level (eg: bundle2) So we move the running of the hooks after transaction creation. This has no visible effects (but an empty transaction roolback if the hook fails) because nothing had happened in the transaction yet.

addchangegroup: get the `node` argument of `incoming` hook from transaction The transaction is now carrying hook-related informations. So we use it to retrieve the `node` argument. This will also carry around all kinds of other useful informations (like: "are we in a bundle2 processing")

dicthelpers: delete now that they are no longer used

manifest: transpose pair of pairs from diff() It makes more sense for the file nodeids and returned from diff() to be ((n1,fl1),(n2,fl2)) than ((n1,n2),(fl1,fl2)), so change it to the former.

manifest: for diff(), only iterate over files, not flags From manifest.diff(), we return a dict from filename to pairs of pairs of file nodeids and flags (values of the form ((n1,n2),(fl1,fl2))). To create this dict, we currently generate one dict for files (with (n1,n2) values) and one for flags (with (fl1,fl2) values) and then join these dicts. Missing files are represented by None and missing flags by '', but due to the dict joining, the inner pairs themselves can also be None. The only caller, merge.manifestmerge(), then unpacks these values while checking for None values. By inlining the calls to dicthelpers and simplifying it to only iterate over files (ignoring flags-only differences), we can simplify life for our caller.

manifest: repurpose flagsdiff() into (node-and-flag)diff() The manifestdict class already has a method for diff flags between two manifests (presumably because there is no full access to the private _flags field). The only caller is merge.manifestmerge(), which also wants a diff of files between the same manifests. Let's combine the code for diffing files and flags into a single method on manifestdict. This puts all the manifest diffing in one place and will allow for further simplification. It might also be useful for it to be encapsulated in manifestdict if we later decide to to shard manifests. The docstring is intentionally unclear about missing entries for now.

util: add a file handle wrapper class that does hash digest validation It is going to be used for the remote-changegroup feature in bundle2.

util: add a helper class to compute digests It is going to be used for the remote-changegroup feature in bundle2.

bundle2: merge return values when bundle contains multiple changegroups A bundle2 may contain multiple parts adding changegroups, in which case there are multiple operation records for changegroups, each with its own return value. Those multiple return values are aggregated in a single cgresult value for the whole operation. As can be seen in the associated test case, the situation with hooks is not really the best, but without deeper thoughts and changes, we can't do much better. Hopefully, things will be improved before bundle2 is enabled by default. In the meanwhile, multiple changegroups is not expected to be in widespread use, and even less expected to be used for pushes. Also, not many clients cloning or pulling bundle2 with multiple changesets are not expected to have changegroup hooks anyways.

changegroup: use a copy of hookargs when invoking the changegroup hook addchangegroup creates a runhook function that is used to invoke the changegroup and incoming hooks, but at the time the function is called, the contents of hookargs associated with the transaction may have been modified externally. For instance, bundle2 code affects it with obsolescence markers and bookmarks info. It also creates problems when a single transaction is used with multiple changegroups added (as per an upcoming change), whereby the contents of hookargs are that of after adding a latter changegroup when invoking the hook for the first changegroup.

tests: pull common http server setup out of individual tests There are currently two different tests using roughly the same code to create temporary scripts acting as HTTP servers. As there is going to be at least one more in an upcoming change, factor those out in a standalone dumbhttp.py script.

util: move md5 back next to sha1 and allow to call it without an argument This effectively backs out changeset 908c5906091b. The API change is done so that both util.sha1 and util.md5 can be called the same way. The function is moved in order to use it for md5 checksumming for an upcoming bundle2 feature.

test-bundle2-exchange: do not drop HG_NODE from the hook output HG_NODE is precisely something we would like to test.

obsolete: update test-commit-amend to use obsolete option This test actually used the obs.py file as part of the test, so we need to fix up the test a little more than usual to work with the new obsolete option flags.

obsolete: update tests to use obsolete options The obsolete._enabled flag has become a config option. This updates all but one of the tests to use the minimal number of flags necessary for them to pass. For most tests this is just 'createmarkers', for a couple tests it's 'allowunstable', and for even fewer it's 'exchange'.

obsolete: prevent options from being used without createmarkers exchange and allowunstable should only be enabled if createmarkers is enabled, so check for that and raise an exception if that's not the case.

obsolete: add exchange option This adds an option that enables obsolete marker exchange.

obsolete: add allowunstable option This option allows the creation of unstable commits. This allows things like amending in the middle of a stack of commits, etc.

obsolete: add createmarkers option The basic obsolete option is allowing the creation of obsolete markers. This does not enable other features, such as allowing unstable commits or exchanging obsolete markers.

obsolete: add readonly flag to obstore constructor Previously, obstore read the obsolete._enabled flag to determine whether to allow writes to the obstore. Since obsolete._enabled will be moving into a repo specific config, we can't read it globally, and therefore must pass the information into the constructor.

obsolete: add isenabled function for option checking Previously, obsolete used the module level _enabled flag to determine whether it was on or off. We need a bit more granular control, so we'll be introducing toggle options. The isenabled() function is how you check if a particular option is enabled for the given repository. Future patches will add options such as 'createmarkers', 'allowunstable', and 'exchange' to enable various features of obsolete markers.

obsstore: fix defaultformat option passing The obsstore format passing was not actually being passed to the obsstore. This fixes it.

tests: use $PYTHON instead of hardcoding python This makes running the testsuite with pypy possible.

checklink: always close the NamedTemporaryFile This fixes test-patchbomb.t when using pypy with --pure.

hgignore: ignore the PyCharm workspace folder

revset: better naming of variables containing the value of a single argument Calling them args is not helpful.

manifest: add docstring to text() method

manifest: rename ambiguously-named set to setflag Just makes it a little clearer what this method does.

bookmarks: inform transaction-related hooks that some bookmarks were moved We do not have enough information to provide any finer data, but this is still useful information.

phases: inform transaction-related hooks that a phase was moved We do not have enough information to provide finer data, but this is still useful information.

test-bundle2: also test the argument of the changegroup hook We also track execution of the changegroup hook. The important information here is to make sure the information that the transaction was processing a bundle2 is passed to hook. This will let most hooks disable themselves while waiting for the hook concluding bundle2 processing (the one we discovered to be not called for pull in the previous changesets).

test-bundle2: test that we got appropriate hook called with appropriate data We can notice that this transaction wide hook is only happening during push and it is missing changegroup-related information. We'll want to fix this but this is not what this patch is about.

pull: use `stepsdone` instead of `todosteps` The push process uses a `stepsdone` attribute instead of a `todosteps` one (with the logic swapped). We unify the two process by picking the `stepsdone` version. I feel like `stepsdone` better fits extensions that would want to extend the push exchange process.

pull: make discovery phase extensible We apply the same approach as for push and make the discovery extensible. There is only one user in core right now, but we already know we'll need something smarter for obsmarkers. In fact the evolve extension could use this to cleanly extend discovery. The main motivation for this change is consistency between push and pull.

sshpeer: forward stdout of remote "hg init" to appropriate output channel Otherwise, commandserver channel could be corrupted.

revlog: support importing censored file revision tombstones This change allows a revision log to not fail integrity checks when applying a changegroup delta (eg from a bundle) results in a censored file tombstone. The tombstone is inserted as-is, so future integrity verification will observe the tombstone. Deltas based on the tombstone will also remain correct. The new code path is encountered for *exactly* the cases where _addrevision is importing a tombstone from a changegroup. When committing a file containing the "magic" tombstone text, the "text" parameter will be non-empty and the checkhash call is not executed (and when committing, the node will be computed to match the "magic" tombstone text).

verify: report censored nodes if configured policy is abort

context: handle censored data in an on-disk file context based on config Two possible behaviors are defined for handling censored data: abort, and ignore. When we ignore censored data we return an empty file to callers requesting the file data.

manifest: add fastdelta method to manifestdict This is another step closer to alternate manifest implementations that can offer different hashing algorithms.

manifest: move _search to module level and rename to _msearch The rename is intended to provide a slight hint that it is manifest-specific.

manifest: move manifestdict-to-text encoding to manifest class A future patch will introduce a new format, with a new class.

localrepo: access status fields by name rather than index

subrepo: use separate instances of empty lists in status We do modify the lists that make up the status in several places, so it seems risky to use the same instance of a list for several different status types. Use a separate empty list for each type instead.

summary: make status code more readable In commands.summary(), we currently zip a list of labels with a list of statuses. This means the order of the status list has to match the list of the labels, which in turn means the status elements have to be inserted into specific places in the list. Let's instead group the labels and status data we want to display in a single list of pairs.

strip: make checklocalchanges() return full status tuple By making checklocalchanges() return the full instance of the status class instead of just the first 4 elements of it, we can take advantage of the field names and not require the caller to remember the element indices.

fileset: access status fields by name rather than index

histedit: access status fields by name rather than index

shelve: access status fields by name rather than index

record: access status fields by name rather than index It is safe to pass the full status to patch.diff() since it does its own slicing.

purge: access status fields by name rather than index

largefiles: access status fields by name rather than index

keyword: access status fields by name rather than index

hgcia: access status fields by name rather than index

context: store status class instead of plain tuple in self._status This improves readability a bit by allowing us to refer to statuses by name rather than index.

status: update and move documentation of status types to status class The various status types are currently documented on the dirstate.status() method. Now that we have a class for the status types, it makese sense to document the status types there instead. Only leave the bits related to lookup/unsure in the status() method documentation.

status: update various other methods to return new class

status: create class for status lists Callers of various status() methods (on dirstate, context, repo) get a tuple of 7 elements, where each element is a list of files. This results in lots of uses of indexes where names would be much more readable. For example, "status.ignored" seems clearer than "status[4]" [1]. So, let's introduce a simple named tuple containing the 7 status fields: modified, added, removed, deleted, unknown, ignored, clean. This patch introduces the class and updates the status methods to return instances of it. Later patches will update the callers. [1] Did you even notice that it should have been "status[5]"? (tweaked by mpm to introduce the class in scmutil and only change one user)

lfutil: avoid creating unnecessary copy of status tuple In lfdirstatestatus(), the status tuple gets deconstructed, the lists get updated, and then an identical status tuple gets created and returned. Change it so we simply return the original tuple.

dirstate: separate 'lookup' status field from others The status tuple returned from dirstate.status() has an additional field compared to the other status tuples: lookup/unsure. This field is just an optimization and not something most callers care about (they want the resolved value of 'modified' or 'clean'). To prepare for a single future status type, let's separate out the 'lookup' field from the rest by having dirstate.status() return a pair: (lookup, status).

commit: update file nodeid and flags in the same place Now that we have a separate variable for the original 'm1' manifest, we can safely update the nodeid of the file in the new manifest in the same place as we update the flags.

commit: use separate variable for p1 manifest and new manifest In localrepo.commitctx(), p1's manifest is copied and used as the basis for the manifest that is about to be committed. The way the copy is updated makes it safe to use it where the original p1's manifest is wanted. For readability, though, a separate variable for each purpose would be clearer. Make it so.

commit: remove dead initialization of 'lock' The 'lock' variable is initialized to None, but before it's ever read, it's assigned again.

commit: reduce scope of 'removed' variable The variable is closely related to 'added' and 'changed', so it makes sense to have it declared next to them.

rebase: fix some weird mixed-case naming

rebase: move duplicatecopies next to merge This is preparation for removing open-coded rebase/graft operations. As a side-effect, this exposes proper renames in the working copy when there are conflicts, which shows up in test-shelve.t.

histedit: use merge.graft

graft: use merge.graft

merge: add merge.graft helper This will help unify all the open-coded graft/rebase operations.

duplicatecopies: move from cmdutil to copies This is in preparation for moving its primary caller into merge.py, which would be a layering violation in the current location.

histedit: fix indent The duplicatecopies call should be part of the rebase block.

graft: move rebase cleanup code next to actual rebase This is prep for refactoring the rebase logic.

shelve: add a bundlerepo method

dirstate: merge falls through to otherparent This lets us more correctly fix the state when we use setparents, as demonstrated in the change in test-graft.t.

dirstate: use 'm' state in otherparent to reduce ambiguity In rebase-like operations where we abandon the second parent, we can correctly fix up the state in setparents.

dirstate: properly clean-up some more merge state on setparents

phases: move root phase assignment to it's own function This moves the initial root phase assignment to it's own function. Future patches which make phase calculations lazy will use this function to pre-fill certain phases which can be deduced from the roots.

phases: add invalidate function Phase cache invalidation was spread all over the place. Let's add a function to unify it. Later more will be added to this function.

phases: change phase command change detection A future patch is going to make phase computation lazy, so the phase command can no longer read and diff the entire phase list directly. This changes the phase command to build it's own list for diff purposes.

spanset: remove `.set()` definition All my friends are dead.

generatorset: remove `.set()` definition All my friends are dead.

addset: remove `.set()` definition All my friends are dead.

filteredset: remove `.set()` definition All my friends are dead.

baseset: remove `set()` definition All my friends are dead.

abstractsmartset: remove `set()` method definition Now that all usages have been removed, we can drop this not so useful part of the API. We can note that the name was wrong all along...

match: check if an object is a baseset using `isascending` instead of `set` The `set()` method is going away.

getset: check if an object is a baseset using `isascending` instead of `set` The `set()` method is going away.

fullreposet: detect smartset using "isascending" instead of "set" The `.set()` function is going away.

fullreposet: drop custom sets but not smartsets detection All custom classes use by revsets are smartsets now. We drop the special-casing.

addset: drop `.set()` usage during iteration We can use the containment check directly.

baseset: access `_set` directly for containment check The `.set()` method is going away.

baseset: make `_set` a property cache This will remove the need for `baseset.set()`.

graphnode: remove usage of `set()` All smartset classes have fast lookup, so this function will be removed soon.

revset-_hexlist: remove usage of `set()` All smartset classes have fast lookup, so this function will be removed soon.

revset-_intlist: remove usage of `set()` All smartset classes have fast lookup, so this function will be removed soon.

revset-_list: remove usage of `set()` All smartset classes have fast lookup, so this function will be removed soon.

revset-roots: remove usage of `set()` All smartset classes have fast lookup, so this function will be removed soon.

histedit: stabilise the order nodes that are stripped The `nodes` object is a set. We sort it to get stable order. This is going to prevent revsets from getting confused when removing a `.set()` call in `roots`.

revset-origin: remove usage of `set()` All smartset classes have fast lookup, so this function will be removed soon.

revset-last: remove usage of `set()` All smartset classes have fast lookup, so this function will be removed soon.

revset-limit: remove usage of `set()` All smartset classes have fast lookup, so this function will be removed soon.

revset-destination: remove usage of `set()` All smartset classes have fast lookup, so this function will be removed soon.

revset-children: remove usage of `set()` All smartset classes have fast lookup, so this function will be removed soon.

revset-branch: remove usage of `set()` All smartset classes have fast lookup, so this function will be removed soon.

revset-rangeset: remove usage of `set()` All smartset classes have fast lookup, so this function will be removed soon.

revset-only: remove usage of `set()` All smartset classes have fast lookup, so this function will be removed soon.

revset: cache most conditions used in `filter` Except when stated otherwise, the condition used in `smartset.filter` will be cached. A new argument has been introduced to disable that behavior. We use it for filters created from `and` and `sub` operations. This gives massive performance boosts for revsets with expensive conditions. revset: branch(stable) or branch(default) before) wall 4.329070 comb 4.320000 user 4.310000 sys 0.010000 (best of 3) after) wall 2.356451 comb 2.360000 user 2.330000 sys 0.030000 (best of 4) revset: author(mpm) or author(lmoscovicz) before) wall 4.434719 comb 4.440000 user 4.440000 sys 0.000000 (best of 3) after) wall 2.321720 comb 2.320000 user 2.320000 sys 0.000000 (best of 4)

baseset: empty or one-element sets are ascending and descending The empty set is full of interesting properties. In the ordering case, the one element set is too.

filteredset: drop explicit order management Now that all low-level smartset classes have proper ordering and fast iteration management, we can just rely on the subset in filteredset.

revset: restore order of `or` operation as in Mercurial 2.9 Lazy revset broke the ordering of the `or` revset. We now stop assuming that two ascending revset are combine into an ascending one. Behavior in 3.0: 3:4 or 2:5 == [2, 3, 4, 5] Behavior in 2.9: 3:4 or 2:5 == [3, 4, 2, 5] We are adding a test for it. For unclear reason, the performance `or` revset with expensive filter are getting even worse than they used to be. This is probably caused by extra uncached containment check or iteration. revset #9: author(lmoscovicz) or author(mpm) before) wall 3.487583 comb 3.490000 user 3.490000 sys 0.000000 (best of 3) after) wall 4.481486 comb 4.480000 user 4.470000 sys 0.010000 (best of 3) revset #10: author(mpm) or author(lmoscovicz) before) wall 3.164839 comb 3.170000 user 3.160000 sys 0.010000 (best of 3) after) wall 4.574965 comb 4.570000 user 4.570000 sys 0.000000 (best of 3)

revset-_descendant: rework the whole sorting and combining logic We use the & operator to combine with subset (since this is more likely to be optimised than filter) and we enforce the sorting of the result. Without this enforced sorting, we may result in a different iteration order than the set _descendent was computed from. This reverts a bad `test-glog.t` change from 69402eb72115. Another side effect is that `test-mq.t` shows `qparent::` including `-1` if `qparent is -1`. This sound like a positive change. This has good and bad impacts on the benchmarks, here is a good ones: revset: 0:: before) wall 0.045489 comb 0.040000 user 0.040000 sys 0.000000 (best of 100) after) wall 0.034330 comb 0.030000 user 0.030000 sys 0.000000 (best of 100) revset: roots((0::) - (0::tip)) before) wall 0.134090 comb 0.140000 user 0.140000 sys 0.000000 (best of 63) after) wall 0.128346 comb 0.130000 user 0.130000 sys 0.000000 (best of 69) revset: ::p1(p1(tip)):: before) wall 0.143892 comb 0.140000 user 0.140000 sys 0.000000 (best of 55) after) wall 0.124502 comb 0.130000 user 0.130000 sys 0.000000 (best of 65) revset: roots((0:tip)::) before) wall 0.204966 comb 0.200000 user 0.200000 sys 0.000000 (best of 43) after) wall 0.184455 comb 0.180000 user 0.180000 sys 0.000000 (best of 47) Here is a bad one: revset: (20000::) - (20000) before) wall 0.009592 comb 0.010000 user 0.010000 sys 0.000000 (best of 222) after) wall 0.029837 comb 0.030000 user 0.030000 sys 0.000000 (best of 100)

addset: do lazy sorting The previous implementation was consuming the whole revset when asked for any sort. The addset class is now doing lazy sorting like all other smarset classes. This has no significant impact in the benchmark as-is. But this is important to later change.

test-import.t: use proper revset order This test, written after 3.0, is relying on addset being enforced ascending if both side are ascending. We are about to restore the ordering to 2.9 behavior (elements are ordered in the order they are specified). We fix the test before fixing the order.

baseset: drop custom __sub__ method This add method is enforcing non-laziness, disabling multiple optimisations. Benchmarks do not spot any significant difference but real usecase may. This will also be important for further improvements to addset later in this series.

baseset: drop custom __and__ method This add method is enforcing non-laziness, disabling multiple optimisations. Benchmarks do not spot any significant regression but real usecase may. This even gives some speedup in some cases: revset #15: min(0::) before) wall 0.001247 comb 0.000000 user 0.000000 sys 0.000000 (best of 1814) after) wall 0.000942 comb 0.000000 user 0.000000 sys 0.000000 (best of 2367) This will also be important for further improvement to addset later in this series.

baseset: drop custom __add__ method This add method is enforcing non-laziness, disabling multiple optimisations. Benchmarks do not spot any significant differences but real usecase may. This will also be important for further improvements to addset later in this series.

obsolete: use format version 1 as the default for obsstore

test-obsolete: remove subminute timezone in test Obsmarker format "1" does not supports sub minute timezone. So we change the test to something slightly more sensible.

obsolete: add a "format.obsstore-version" config option This option controls what version of the binary format to use when creating a new obsstore file. Default is still the old format. No safeguards are currently placed around the option value, but no clueless users are in danger of harm since it is undocumented.

obsolete: introduce a new binary encoding for obsmarkers (version 1) This new encoding explicitly stores the date and parents allowing a significantly faster marker decoding. See inline documentation for details. This format is not yet used to store format on disk. But it will be used in bundle2 exchange if both side support it. Support for on-disk format is coming in other changesets.

obsstore: add a flag for sha256 hashes We add flag to inform that the marker is using sha256 hashes. As format 0 is not able to handle sha256 hashes (32 bytes long), we plain crash if we even attempt to encode a sha256 with it.

obsolete: use uint## in the format documention This is shorter and kind of more readable for people who care about binary format.

obsolete: gather _fm0 meta encoding with other _fm0 code

obsolete: _rename decodemeta to _fm0decodemeta This will be format zero specific.

obsolete: _rename encodemeta to _fm0encodemeta This will be format zero specific.

obsolete: store metadata as a tuple of (key, value) pairs (API) Different formats will encode metadata in different ways. So we cannot keep the binary blob in the object anymore. We use a tuple to ensure it is immutable and hashable.

merge with stable

templater: fix ifcontains when list is a string (issue4399)

shelve: don't delete "." when rebase is a no-op (issue4398) When unshelving and facing a conflict, if we resolve all conflicts in favour of the committed changes instead of the shelved changes, then the ensuing implicit rebase is a no-op. That is, there is nothing to rebase. In this case, there are no extra intermediate shelve commits to strip either. Prior to this change, the commit being unshelved to would be marked for destruction in a rather catastrophic way. The relevant part of the test case failed as follows: $ hg unshelve -c unshelve of 'default' complete $ hg diff warning: ignoring unknown working parent 33f7f61e6c5e! diff --git a/a/a b/a/a new file mode 100644 --- /dev/null b/a/a @@ -0,0 1,3 @@ a c x $ hg status warning: ignoring unknown working parent 33f7f61e6c5e! M a/a ? a/a.orig ? foo/foo $ hg summary warning: ignoring unknown working parent 33f7f61e6c5e! parent: -1:000000000000 (no revision checked out) branch: default commit: 1 modified, 2 unknown (new branch head) update: 4 new changesets (update) With this change, this test case now passes.

merge: make error message consistent with other commands If a merge is attempted when another merge is already ongoing, we give the message "outstanding uncommitted merges". Many other commands (such as backout, rebase, histedit) give the same message in singular form. Since the singular form also seems to make more sense, let's use that for 'hg merge' as well.

test-run-tests: add a test for detection of failure to start a server This also highlights a bug: right now we print "2 failed" but we only ran one test.

run-tests: more accurate/helpful message than "diff generation failed" Diff generation didn't really fail, it recognized that an hg serve server has failed to start, and thus skipped the diff generation intentionally. The most common reason for a server to fail to start is that the port was already in use, so output HGPORT as well, to help finding it (since pgrep -f 'hg serve' is not sufficient, if the command line is something like 'hg -R main serve')

run-tests: handle --jobs and --first gracefully Without this change, --first causes currently-running tests to explode in violent and surprising ways when their temporary directory gets cleaned up. Now we just suppress failure messages from non-first failures when running in --first mode.

config: use the same hgrc for a cloned repo as for an uninitted repo This just copies the same local sample hgrc, except it sets the default path to the repo it was cloned from. This is cut-and-paste from the local sample hgrc, but I think it's acceptable, since the two pieces of code are right next to each other and they're small. There is danger of them going out of synch, but it would complicate the code too much to get rid of this C&P. I also add ui as an import to hg.py, but with demandimport, this should not be a noticeable performance hit.

config: give a more detailed sample repo config Some examples of the typical configurations that one might want to do in an .hg/hgrc file. This includes a default-push that happens to point to the same location as my-fork. I insist on the myfork terminology for a server-side clone. Bitbucket, Github, and others have widely popularised this meaning of "fork". This also includes a gentle nudge to use a repo-specific username, which is something that people might not instinctively realise is an option.

smartset: drop infamous ascending, descending All your friends are dead.

fullreposet: use `isascending` instead of `ascending` to recognise smartsets `ascending` is going to be removed.

fullreposet: use `sort` to enforce the order The `ascending` and `descending` methods are useless.

revancestors: replace `descending` with `sort(reverse=False)`

_descendants: replace `ascending()` with `sort()`

_descendants: directly use smartset As `addset` objects are proper smartset objects, we do not need to make any transformation of the result.

baseset: explicitly track order of the baseset A baseset starts without an explicit order. But as soon as a sort is requested, we simply register that the baseset has an order and use the ordered version of the list to behave accordingly. We will want to properly record the order at creation time in the future. This would unlock more optimisation and avoid some sorting.

baseset: fix isascending and isdescending We now have sufficient information to return the proper value there.

baseset: prepare lazy ordering in __iter__ We'll explicitly track the order of the baseset to take advantage of the ascending and descending lists during iteration.

baseset: implement a fastasc and fastdesc Baseset contains already-computed revisions. It is considered "cheap" to do operations on an already-computed set. So we add attributes to hold version of the list in ascending and descending order and use them for `fastasc` and `fastdesc`. Having distinct lists is important to provide correct iteration in all cases. Altering a python list will impact an iterator connected to it. eg: not preserving order at iterator creation time >>> l = [0, 1] >>> i = iter(l) >>> l.reverse() >>> list(i) [1, 0] eg: corrupting in progress iteration >>> l = [0, 1] >>> i = iter(l) >>> i.next() 0 >>> l.reverse() >>> i.next() 0

baseset: stop inheriting from built-in list class The baseset is doing more and more smartset magic and using its list-like property less and less. So we store the list of revisions in an explicit attribute and stop inheriting. This requires reimplementing some basic methods.

strip: stop calling `remove` on smartset The `remove` method is not part of the smartset specification. We use a plain old list comprehension instead.

rebase: transform the smartset to a list before comparing with a list This is highly suboptimal but smartsets are not comparable to lists yet.

merge.update: use `first` instead of direct indexing This makes it compatible with all smartset classes.

qimport: use `first` and `last` instead of direct indexing This makes it compatible with all smartset classes.

rebase: use `last` instead of direct indexing This makes it compatible with all smartset classes.

mq: use `last` instead of direct indexing This makes it compatible with all smartset classes.

repair: use `first` instead of direct indexing This makes it compatible with all smartset classes.

rangeset: use `first` and `last` instead of direct indexing This makes it compatible with all smarsets classes.

revpair: use `first` and `last` instead of direct indexing This makes it compatible with all smarsets classes.

revsingle: use `last` instead of direct indexing This makes it compatible with all smarsets classes.

revset-limit: use boolean testing instead of `len(revs) < 1` I'm not sure why we wrote it that way. But smartsets have faster/lazier non-zero testing than length computation.

filteredset: implement `first` and `last`

baseset: implement `first` and `last` methods

generatorset: implement first and last methods

addset: implement first and last methods The implementation is non-lazy for now. One may want to make it more lazy in the future.

spanset: implement `first` and `last` methods

smartset: add first and last methods In multiple places in the code, we use `someset[0]` or `someset[-1]`. This works only because the `someset` is usually a baseset. For the same reason we introduce a `first` and `last` methods to be implemented for all smartset classes.

getgraphlogrevs: remove user of baseset.append A `baseset` has multiple cached results and will get even more in the future. Making it an object "populated once" like the other smartsets makes it both safer and simpler. The append method will be removed at some point.

getlogrevs: remove user of baseset.append A `baseset` has multiple cached results and will get even more in the future. Making it an object "populated once" like the other smartsets makes it both safer and simpler. The append method will be removed at some point.

revset-last: remove user of baseset.append A `baseset` has multiple cached results and will get even more in the future. Making it an object "populated once" like the other smartsets makes it both safer and simpler. The append method will be removed at some point.

revset-limit: remove user of baseset.append A `baseset` has multiple cached results and will get even more in the future. Making it an object "populated once" like the other smartsets makes it both safer and simpler. The append method will be removed at some point.

mq: use `revs.sort()` to ensure the set is ascending Sorting is super-cheap with the new smartset class, so we can use it to enforce the order. Otherwise all smartset classes would have to allow direct indexing.

baseset: use default value instead of [] when possible For pure cleanup purposes, we replace all the occurences of `baseset([])` with `baseset()`.

generatorset: implement isascending and isdescending

generatorset: explicitly track iteration order The expected iteration order may be different than the fast iteration order (eg: ancestors(42) is expected to be iterated upward but is fast/lazy to compute downward. So we explicitly track the iteration order and enforce it if the manual iteration is requested. Default expected iteration order of a generator set is ascending because I'm not aware of any descending revset that need a generatorset. The first to find such descending revset will have the pleasure to make this configurable.

addset: drop caching through generatorset The utility of this cache is debatable (no visible benchmark impact) and using generatorset for such purpose makes the code complicated. We drop it for now. Someone can reintroduce a smart version of it in the future if it is detected to be relevant.

generatorset: get list-based fast iterations after the generator is consumed When all revisions are known, we shortcut most of the class logic to use list iteration instead. The cost of the sort is expected to be non-significant. The list creation and sorting could be done lazily in the future. We have to copy the list to not break existing iterator created before we finished consuming the generator.

generatorset: move iteration code into _iterator _iterator handles the generator iteration. The `__iter__` method will need changes to handle ordering-related information.

generatorset: stop using a base as the _genlist It does not add anything and makes it more complicated to have a simple baseset implementation.

generatorset: drop the leading underscore in the class name This is a real smart set now.

generatorset: update the docstring now that it is a smartset The documentation was still stating that this class was not a smartset. We drop that part.

addset: drop the leading underscore from the class name This class is now a real smartset.

addset: this is a smartset, update the docstring The documentation was still stating that this class is a not a smartset. We drop that part.

addset: use the ascending argument in _iterordered Fix a bug where fastasc and fastdesc were iterator in the same order as self._ascending.

rebase: add help examples

rebase: attempt to clarify --base

manifest: rearrange add() method and add comments for clarity Omit the check of bool(p1) since it's always true in practice: it will either be nullid or some valid manifest sha, and we know nullid won't ever be in the cache so we can simplify understanding of this code.

manifest: simplify manifest.add() by making args required I verified that changed was never false (it was always a 2-tuple) by adding @@ -220,6 +225,8 @@ class manifest(revlog.revlog): def add(self, map, transaction, link, p1=None, p2=None, changed=None): + if not changed: + assert False, 'changed was %r' % changed # if we're using the cache, make sure it is valid and # parented by the same node we're diffing against if not (changed and p1 and (p1 in self._mancache)): and observing that the test suite still passed. Making all the arguments required should help future readers understand what's going on here.