histedit: also handle locally missing nodes when reading obsolescence The previous version of the code was interpreting markers to a missing node as a prune in all cases. The expected way to handle such situation is to keep reading markers, only turning successors into "prune" if they are at the end of a chain. We update the code and add a test for this.

chgserver: auto exit after being idle for too long or lose the socket file This is a part of the one server per config series. In multiple-server setup, new server may be started for a temporary config change like in command line, --config extensions.foo=bar.py. This may end up with a lot of not so useful server processes. Other questions are about socket file and process management, How to stop these processes? What if a new server wants to listen on a same address, replacing the old one? This patch introduces AutoExitMixIn, which will: 1. Exit after being idle for too long. So useless servers won't run forever. 2. Periodically check the ownership of socket file, exit if it is no longer owned. This brings strong consistency between the filesystem and the process, and handles some race conditions neatly. Since rename is atomic, a new server can just have a same server address with an old one and won't worry about how to make sure the old server is killed, address conflict, service downtime issues. The user can safely stop all servers by simply removing the socket files, without worrying about outdated or accidentally removed pidfiles.

unionrepo: properly handle hidden linkrev in revlog (issue5070) The unionrepository have to do some special magic to handle linkrev of the unioned filerev and manifestrev. That logic was done from a repoview and obsolescence marker affecting bundled changeset could lead to a crash. We now ensure we operate on unfiltered repository.

bundlerepo: properly handle hidden linkrev in manifestlog (issue4945) The bundlerepository have to do some special magic to handle linkrev of the bundled manifest. That logic was done from a repoview and obsolescence marker affecting bundled changeset could lead to a crash. We now ensure we operate on unfiltered repository.

bundlerepo: properly handle hidden linkrev in filelog (issue4945) The bundlerepository have to do some special magic to handle linkrev of the bundlerepo filerev. That logic was done from a repoview and obsolescence marker affecting bundled changeset could lead to a crash. We now ensure we operate on unfiltered repository.

check-code: allow old style class with special comments The following chgserver change will use an old style class to comply with SocketServer's code style. This patch made it possible to pass check-code.

merge with stable

revset: use smartset minus operator Previously, revsets like 'X - Y' were translated to be 'X and not Y'. This can be expensive, since if Y is a single commit then 'not Y' becomes a huge set and sometimes the query optimizer doesn't account for it well. This patch changes revsets to use the built in smartset minus operator, which is often smarter than 'X and not Y'. On a large repo this saves 2.2 seconds on rebase and histedit because "X:: - X" becomes almost instant. Relevant performance numbers from revsetbenchmark.py revset #13: roots((tip~100::) - (tip~100::tip)) plain min max first last reverse rev..rst rev..ast sort sor..rst sor..ast 0) 0.001080 0.001107 0.001102 0.001118 0.001121 0.001114 0.001141 0.001123 0.001099 0.001123 0.001137 1) 0.000708 65% 0.000738 66% 0.000735 66% 0.000739 66% 0.000784 69% 0.000780 70% 0.000807 70% 0.000756 67% 0.000727 66% 0.000759 67% 0.000808 71% revset #14: roots((0::) - (0::tip)) plain min max first last reverse rev..rst rev..ast sort sor..rst sor..ast 0) 0.131304 0.079168 0.133129 0.076560 0.048179 0.133349 0.049153 0.077097 0.129689 0.076212 0.048543 1) 0.065066 49% 0.036941 46% 0.066063 49% 0.034755 45% 0.048558 0.071091 53% 0.047679 0.034984 45% 0.064572 49% 0.035680 46% 0.048508 revset #22: (not public() - obsolete()) plain min max first last reverse rev..rst rev..ast sort sor..rst sor..ast 0) 0.000139 0.000133 0.000133 0.000138 0.000134 0.000155 0.000157 0.000152 0.000157 0.000156 0.000153 1) 0.000108 77% 0.000129 0.000129 0.000134 0.000132 0.000127 81% 0.000151 0.000147 0.000127 80% 0.000152 0.000149 revset #25: (20000::) - (20000) plain min max first last reverse rev..rst rev..ast sort sor..rst sor..ast 0) 0.050560 0.045513 0.022593 0.043588 0.021909 0.045517 0.021822 0.044660 0.049740 0.044227 0.021819 1) 0.018614 36% 0.000171 0% 0.019659 87% 0.000168 0% 0.015543 70% 0.021069 46% 0.015623 71% 0.000180 0% 0.018658 37% 0.000186 0% 0.015750 72%

histedit: make histedit aware of obsolescense not stored in state (issue4800) Before this change, when histedit exited to interactive session (during edit command for example), user could introduce obsolescence markers that would not be known to histedit. For example, user could've amended one of the commits. The fact of this amendment would not be stored in histedit's state file and later, when histedit would try to process all the replacements, one of the final successors (in histedit's opinion) would turn out to be hidden. This behavior is described in issue4800. This commit fixes it.

treemanifest: allow setting flag to 't' When using treemanifests, an on-disk manifest entry with the 't' flag set means that that entry is a directory and not a file. When read into memory, these become instances of the treemanifest class. The 't' flag should therefore never be visible to outside of manifest.py, so setflag() checks that it is not called with the 't' flag. However, it turns out that it will be useful for the narrowhg extension to expose the 't' flag to the user (see below), so let's drop the assertion. The narrowhg extension allows cloning only a given set of files and directories. Filelogs and dirlogs that don't match that set will not be included in the clone. The extension currently doesn't work with treemanifests. I plan on changing it so directories outside the narrow clone appear in the manifest. For example, if a directory 'outside/' is not part of the narrow clone, it will look like a file 'outside' with the 't' flag set. That will make e.g. manifestmerge() just work in most cases (and make it well prepared to handle the other cases).

treemanifest: use "cp xyz/." instead of "cp xyz/*" This is more similar to cp -T because it covers hidden files.

templatefilters: drop old jsonescape() function It's been superseded by encoding.jsonescape(paranoid=True).

templatefilters: make json filter be byte-transparent (BC) (issue4926) This is necessary to preserve filename encoding over JSON. Instead, this patch inserts "|utf8" where non-ascii local-encoding texts can be passed to "|json". See also the commit that introduced "utf8" filter.

hgweb: add option to convert encoding of graphdata() Because future patches will change "|json" filter to handle input bytes transparently, i.e. use UTF-8b encoding, "{jsdata}" must keep data in UTF-8 bytes, whereas "{nodes}" are text. This patch inserts encodestr() where localstr is likely to survive.

hgweb: remove unused argument from graphdata() factory As graphdata() is wrapped by lambda, there's no reason to pass unused arguments to it.

templatefilters: add "utf8" to get utf-8 bytes from local-encoding text This will be applied prior to "|json" filter. This sounds like odd, but it is necessary to handle local-encoding text as well as raw filename bytes. Because filenames are bytes in Mercurial and Unix world, {filename|json} should preserve the original byte sequence, which implies {x|json} -> '"' toutf8b(x) '"' On the other hand, most template strings are in local encoding. Because "|json" filter have to be byte-transparent to filenames, we need something to annotate an input as a local string, that's what "|utf8" will do. {x|utf8|json} -> '"' toutf8b(fromlocal(x)) '"' "|utf8" is an explicit call, so aborts if input bytes can't be converted to UTF-8.

templatefilters: drop broken "jsonescape" from filters table (BC) It's been unused, undocumented and flawed in that it expects a unicode input, never works correctly if an input has non-ascii character. We should use "json" filter instead.

treemanifest: rewrite text() using iterentries() This simplifies a bit. Note that the function is only used when manually testing with _treeinmem=True.

treemanifest: implement iterentries() To make tests pass with _treeinmem manually set to True, we need to implement the recently added iterentries() on the treemanifest class too.

verify: show progress while verifying dirlogs In repos with treemanifests, the non-root-directory dirlogs often have many more total revisions than the root manifest log has. This change adds progress out to that part of 'hg verify'. Since the verification is recursive along the directory tree, we don't know how many total revisions there are at the beginning of the command, so instead we report progress in units of directories, much like we report progress for verification of files today. I'm not very happy with passing both 'storefiles' and 'progress' into the recursive calls. I tried passing in just a 'visitdir(dir)' callback, but the results did not seem better overall. I'm happy to update if anyone has better ideas.

verify: check for orphaned dirlogs We already report orphaned filelogs, i.e. revlogs for files that are not mentioned in any manifest. This change adds checking for orphaned dirlogs, i.e. revlogs that are not mentioned in any parent-directory dirlog. Note that, for fncachestore, only files mentioned in the fncache are considered, there's not check for files in .hg/store/meta that are not mentioned in the fncache. This is no different from the current situation for filelogs.

verify: check directory manifests In repos with treemanifests, there is no specific verification of directory manifest revlogs. It simply collects all file nodes by reading each manifest delta. With treemanifests, that's means calling the manifest._slowreaddelta(). If there are missing revlog entries in a subdirectory revlog, 'hg verify' will simply report the exception that occurred while trying to read the root manifest: manifest@0: reading delta 1700e2e92882: meta/b/00manifest.i@67688a370455: no node This patch changes the verify code to load only the root manifest at first and verify all revisions of it, then verify all revisions of each direct subdirectory, and so on, recursively. The above message becomes b/@0: parent-directory manifest refers to unknown revision 67688a370455 Since the new algorithm reads a single revlog at a time and in order, 'hg verify' on a treemanifest version of the hg core repo goes from ~50s to ~14s. As expected, there is no significant difference on a repo with flat manifests.

hg: perform update after pulling during clone with share (issue5103) When pooled storage is enabled, `hg clone` will initialize a repo from a local repo using the store sharing mechanism then pull from the originally requested repo. Before this patch, the working directory update occurred between these steps. This meant that we would only update to revisions that were already present in the local pooled storage. This patch moves the update to after we pull from the originally requested repository so we may check out a revision that didn't yet exist locally. In other words, it makes the behavior like normal `hg clone`.

hg: extract post share update logic into own function A future patch will introduce a new caller that needs to perform an update. Extract the code so we don't duplicate it.

merge: perform background file closing in batchget As 2fdbf22a1b63 demonstrated with stream clones, closing files on background threads on Windows can yield a significant speedup because closing files that have been created/appended to is slow on Windows/NTFS. Working directory updates can write thousands of files. Therefore it is susceptible to excessive slowness on Windows due to slow file closes. This patch enables background file closing when performing working directory file writes. The impact when performing an `hg up tip` on mozilla-central (136,357 files) from an empty working directory is significant: Before: 535s (8:55) After: 133s (2:13) Delta: -402s (6:42) That's a 4x speedup! By comparison, that same machine can perform the same operation in ~15s on Linux. So Windows went from ~35x to ~9x slower. Not bad but there's still work to do. As a reminder, background file closing is only activated on Windows because it is only beneficial on that platform. So this patch shouldn't change non-Windows behavior at all. It's worth noting that non-Windows systems perform working directory updates with multiple processes. Unfortunately, worker.py doesn't yet support Windows. So, there is still plenty of room for making working directory updates faster on Windows. Even if multiple processes are used on Windows, I believe background file closing will still provide a benefit, as individual processes will still be slowed down by the file close bottleneck (assuming the I/O system isn't saturated).

merge: indent code in batchget() To make the next patch easier to read.

localrepo: support background closing for wwrite() So working copy update can pass it in.

scmutil: support background closing for write() Upcoming patches will add background file closer support to working copy update. This patch adds some plumbing to prepare for that.

chg: hold a lock file before connected to server This is a part of the one server per config series. In multiple-server setup, multiple clients may try to start different servers (on demand) at the same time. The old lock will not guarantee a client to connect to the server it just started, and is not crash friendly. This patch addressed above issues by using flock and does not release the lock until the client actually connects to the server or times out.

serve: allow --daemon-postexec to be 'unlink:path' or 'none' This patch changes the format of value of --daemon-postexec. Now it can be either to unlink a file, or a no-op. The following patch will make chg to use the no-op option.

serve: rename --daemon-pipefds to --daemon-postexec (BC) Initially we use --daemon-pipefds to pass file descriptors for synchronization. Later, in order to support Windows, --daemon-pipefds is changed to accept a file path to unlink instead. The name is outdated since then. chg client is designed to use flock, which will be held before starting a server and until the client actually connects to the server it started. The unlink synchronization approach is not so helpful in this case. To address the issues, this patch renames pipefds to postexec and the following patch will allow the value of --daemon-postexec to be things like 'unlink:/path/to/file' or 'none'.

largefiles: don't explicitly list optional parameters that are not used This makes it easier for changes to the API.

revert: properly revert to ancestor of p2 during merge (issue5052) During merge, added (from one perspective) file can be reported as "modified". To work around that, revert was testing if modified file were present in the parent manifest and marking them as "added" in this case. However, we should be checking against the target revision manifest instead. Otherwise see file as "newly added" even if they exist in the target revision. That revert behavior regressed in 06fbd9518bc5.

help: hg.intevation.de is new primary name of hg.intevation.de (and new cert) Adjust the examples (prefix and hostfingerprints) in help/config.txt https://hg.intevation.de/ is now served with a new certificate that is signed by a commercial CA, so all nearly all browsers will accept it automatically. Listing both names, hg.intevation.de and hg.intevation.org, in the section [hostfingerprints] allows using both without configuring web.cacerts and without changing existing https URLs in the [paths] section.

rebase: explicitly test abort from ambiguous destination We want to explicitly test the next behavior. We add this test in its own changeset to make the test change from the behavior change as clean as possible.

rebase: choose default destination the same way as 'hg merge' (BC) This changeset finally make 'hg rebase' choose its default destination using the same logic as 'hg merge'. The previous default was "tipmost changeset on the current branch", the new default is "the other head if there is only one". This change has multiple consequences: - Multiple tests which were not rebasing anything (rebasing from tipmost head) are now rebasing on the other "lower" branch. This is the expected new behavior. - A test is now explicitly aborting when there is too many heads on the branch. This is the expected behavior. - We gained a better detection of the "nothing to rebase" case while performing 'hg pull --rebase' so the message have been updated. Making clearer than an update was performed and why. This is beneficial side-effect. - Rebasing from an active bookmark will behave the same as 'hg merge' from a bookmark.

rebase: add potential divergent commit hashes to error message (issue5086)

hgwebdir_wsgi: update script and expand help I've updated the script to reflect changes in Mercurial and to include a much more through installation guide with configuration examples and details on how to configure IIS. I've used the script to set up a working server from scratch.

bundlerepo: properly handle hidden linkrev in filelog (issue4945) The bundlerepository have to do some special magic to handle linkrev of the bundlerepo filerev. That logic was done from a repoview and obsolescence marker affecting bundled changeset could lead to a crash. We now ensure we operate on unfiltered repository.

rebase: adds storing collapse message (issue4792) Before this patch collapse message wasn't stored so when you ran into the merge conflict while rebasing, running rebase --continue didn't remember the message and always opened editor to fill commit message. This patch adds saving collapse message in .hg/last-message.txt and restoring it later when needed.

test-automv: fix inline config settings for 5ec1ce8fdf0a

automv: use 95 as the default similarity threshold The motivation for the change from 100 to 95 is included in a comment. * Updated the tests to include a change to a moved file that still should be caught as a move. * Use ui.configint() to non-integer configuration entries more gracefully. Also complain if a similarity outside of the acceptable range is set.

bookmarks: add 'hg push -B .' for pushing the active bookmark (issue4917)

worker: change partition strategy to every Nth element The only consumer of the worker pool code today is `hg update`. Previously, the algorithm to partition work to each worker process preserved input list ordering. We'd take the first N elements, then the next N elements, etc. Measurements on mozilla-central demonstrate this isn't an optimal partitioning strategy. I added debug code to print when workers were exiting. When performing a working copy update on a previously empty working copy of mozilla-central, I noticed that process lifetimes were all over the map. One worker would complete after 7s. Many would complete after 12s. And another worker would often take >16s. This behavior occurred for many worker process counts and was more pronounced on some than others. What I suspect is happening is some workers end up with lots of small files and others with large files. This is because the update code passes in actions according to sorted filenames. And, directories under tend to accumulate similar files. For example, test directories often consist of many small test files and media directories contain binary (often larger) media files. This patch changes the partitioning algorithm to select every Nth element from the input list. Each worker thus has a similar composition of files to operate on. The result of this change is that worker processes now all tend to exit around the same time. The possibility of a long pole due to being unlucky and receiving all the large files has been mitigated. Overall execution time seems to drop, but not by a statistically significant amount on mozilla-central. However, repositories with directories containing many large files will likely show a drop. There shouldn't be any regressions due to partial manifest decoding because the update code already iterates the manifest to determine what files to operate on, so the manifest should already be decoded.

run-tests: allow run-tests.py to run tests outside current directory When reloading tests, run-tests.py was assuming that it could look up the test by the basename, which only works if you are running tests which are in the current directory. This patch changes that lookup to use the full path. This is all that was needed, and does not appear to cause any problems for any of the existing testing work flows based on running the suggested commands at the top of run-tests.py. Motivation: In order to test Mercurial with Hypothesis (according to https://www.mercurial-scm.org/wiki/HypothesisPlan) it is useful to be able to generate temporary test files and execute them. Generating temporary files in the tests/ directory leads to a lot of suboptimal clutter.

histedit: unifying the way replacements are computed for abort and success This is a part of a bigger refactoring effort with the ultimate goal of better understanding of how histedit works and fixing issue4800.

templater: factor out type conversion of revset() result This makes it clear why we have to do repo[int(x)].

templater: evaluate each item of revset() as integer revision Because templater.revset() returns a list of strings, repo["-1"] was mapped to the tipmost revision. Ideally, we should make revset() return a list of integer revisions, but it turned out not simple. If revset() is a list of integers, "{ifcontains(rev, revset(), ...)}" would fail because "ifcontains" casts "rev" to a string. So this patch just converts a string back to an integer revision.

demandimport: blacklist sqlalchemy.events as it has side effects (issue5085) Importing sqlalchemy.events cannot be delayed as it registers classes to their event mechanism. It worked fine before 4f1144c3c72b, since they use new-style imports. But now we have to blacklist it because our demandimport can handle new-style imports. This patch series isn't intended for stable as we don't guarantee API compatibility with 3rd-party extensions. They can temporarily disable the demand importer to work around the issue.

demandimport: enforce ignore list while processing modules in fromlist If a module is loaded as "from . import x" form, there has been no way to disable demand loading for that module because name is ''. This patch makes it possible to prevent demand loading by '<package-name>.x'. We don't use _hgextimport(_origimport) here since attr is known to be a sub-module name. Adding hgext_ to attr wouldn't make sense.

phases: use constants for phase values Magic numbers hinder readability.

test-paths: fix up json output to match $TESTTMP on Windows Without this, the paths were in the form "C:\\Users\\Matt\\AppData...". The test runner wouldn't collapse it down to $TESTTMP, which uses single backslashes.

test-obsolete: conditionalize a test dependent on chmod

progress: display progress bar when HGPLAINEXCEPT contains "progress" This patch changes "progress.shouldprint()" so a feature name is provided to "ui.plain()" to determine if there is an exception specificed in HGPLAINEXCEPT for the progress extension. This will allow user-facing scripts to provide progress output while HGPLAIN is enabled.

run-tests: stop allocating HGPORT3+HGPORT4 The only consumer was test-treemanifest.t, which has been fixed. In general, you should be able to use killdaemons.py to recycle ports instead of going over 3 ports (HGPORT, HGPORT1, HGPORT2). In the future, if you want to add a port, be sure to change portneeded in _getport.

run-tests: refactor port allocation into functions Adding a port reservation was too hard and someone did it wrong. By refactoring, such reservations can be managed more safely. This also adds documentation so that the next person who tries is more likely to update all the places correctly. Note that in this commit the reservation and consumers do not match, that will be fixed in the next commit.

tests: put test-treemanifest.t on a port diet test-treemanifest.t had introduced HGPORT3 and HGPORT4, which were improperly added to run-tests.py. It also was not using HGPORT1. This recycles HGPORT, and shifts everything into HGPORT1 + HGPORT2.

chg: pass sensitive command line flags to server We are going to make chgserver load repo config, remember what it is, and load repo config again to detect config change. This is the first step that passes config, repo, cwd options to server. Traceback is passed as well to cover errors before hitting chgserver.runcommand.

chg: use mallocx and reallocx in hgclient This patch simplifies the code a bit, and reduces the binary size a little.

chg: add utility functions mallocx, reallocx They are like malloc and realloc but will abort the program on error. A lot of places use {m,re}alloc and check their results. This patch can simplify them.

localrepo: move new repo requirements into standalone function (API) This patch extracts the code for determining requirements for a new repo into a standalone function. By doing so, future code that will perform an in-place repository upgrade (e.g. to generaldelta) can examine the set of proposed new requirements and possibly take additional actions (such as adding dotencode or fncache) when performing the upgrade. This patch is marked as API because _baserequirements (which was added in b090601a80d1 so extensions could override it) has been removed and will presumably impact whatever extension it was added for. Consumers should be able to monkeypatch the new function to achieve the same functionality. The "create" argument has been dropped because the function is only called in one location and "create" is always true in that case. While it makes logical sense for this code to be a method so extensions can implement a custom repo class / method to override it, this won't actually work. This is because requirements determination occurs during localrepository.__init__ and this is before the "reposetup" "callback" is fired. So, the only way for extensions to customize requirements would be to overwrite localrepo.localrepository or to monkeypatch a function on a module during extsetup(). Since we try to keep localrepository small, we use a standalone function. There is probably room to offer extensions a "hook" point to alter repository creation. But that is scope bloat.

localrepo: isolate requirements determination from side effects In preparation for moving requirements determination to its own function.

localrepo: use local variable for requirements assignment A future patch will refactor requirements determination into a standalone function. To prepare for this, refactor the requirements code to assign to a local variable instead of to self.requirements.

destutil: ensure we offer 'hg update' hint when not at head in all cases In the merge case, we abort if the working copy is not at head, offering to run 'hg update' in the hint instead. In the rebase case, we do not abort in that case. Yet if no rebase destination are found, it still make sense to hint the user about running 'hg update'. So we re-introduce a conditional using this branch in the 'onheadcheck == False' case. This will get used on rebase use this function.

chg: forward umask from client to server This is necessary to make chg test pass on test-inherit-mode.t.

chgserver: add setumask method Before this patch, the server won't inherit umask from the client, which will fail test-inherit-mode.t. This patch provides a way for the client to change umask of the server, similar to chdir and setenv.

chgserver: use _readlist and _readstr Use _readlist and _readstr to make the code shorter.

commandserver: use _readlist Use _readlist introduced in previous commit to make the code shorter.

commandserver: add _readstr and _readlist Reading a string or a list are common operations for commandserver and chgserver. This patch adds _readstr and _readlist to avoid duplication.

extensions: add notloaded method to return extensions failed to load Before this patch, there is no easy way to detect if there are extensions failed to load. While this is okay for most situations, chgserver is designed to preload all extensions specified in config and does need the information. This patch adds extensions.notloaded() to return names of extensions failed to load.

histedit: renaming parts to which _histedit was split I recently broke _histedit into multiple functions, each of which had 'action' in its name. This is a little bit confusing since 'action' is the word for an individual histedit plan item (such as edit or pick). To avoid this confusion, these functions are now renamed to contain 'histedit' as part of their names.

histedit: break _histedit into smaller pieces (add _finishaction) This is a part of a bigger effort to refactor histedit with ultimate goal of understanding it.

automv: switch to specifying the similarity as an integer (0-100) This is consistent with the addremove --similarity option.

automv: do not release lock between marking files and the actual commit

automv: reuse existing scutil._markchanges() function

automv: improve function docstrings

automv: remove test for 'developer mode' The tests no longer use this mode, the test is entirely redundant.

automv: add some context for the tests

automv: simplify retrieving the status The default is to show the status in the current working copy, no need to pass in the parent and working copy context here.

tests: change branches test to work cross platform While it is not easy to make a file 000 on Windows, you can emulate most of the behaviors by replacing the file with a directory.

histedit: change string literals to constants in goal naming This is a part of bigger effort to refactor histedit with ultimate goal of understanding it.

run-tests: add --chg option to install and run tests using chg Because the temporary installation directory is shared between hg and chg, --chg is not allowed if --with-hg option is specified. Also, --chg option does not work on FreeBSD because "make" command is hard-coded. These limitations can be improved later. Almost all tests will fail with chg right now.

run-tests: add --with-chg option to run tests using chg Unlike --with-hg=/path/to/chg, this option allows us to start and clean up command servers in isolated environment. And we can specify the hg command as well as the chg command.

destutil: add more precise error classes for destmerge Having exception classes more precise than 'Abort' will allow us to properly catch "nothing to rebase" situations when we will be using 'destmerge' in rebase.

destutil: allow to disable the "on head check" in destmerge 'hg merge' refuses to pick a default destination if the working copy is not on a head. This is a very sensible default for 'hg merge' but 'hg rebase' should work in this situation. So we introduce a way to disable this check. It will soon be used by rebase.

destutil: allow to specify an explicit source for the merge We can now specify from where the merge is performed. The experimental revset is updated to take revisions as argument, allowing to test the feature. This will become very useful for pick the 'rebase' default destination. For this reason, we also exclude all descendants from the rebased set from the candidate destinations. This descendants exclusion was not necessary for merge as default destination would not be picked from anything else than a head. I'm not super excited with the current error messages, but I would prefer to delay an overall messages rework once 'hg rebase' is done getting a default destination aligned with 'hg merge'.

destutil: remove current head from list of candidates early While 'hg merge' will refuse to pick a default destination if the working copy is not on a head, this will be a common and valid case for rebase. In this case, we will need to exclude from the candidate destination all descendants from the rebased set. We make a step forward in that direction by removing parents of the working copy from the candidate destinations instead of manually filtering the working copy parent at the end of the process. This will make the extra step of filtering descendant much simpler in a future changeset.

destutil: add an 'action' layer to the destmerge message dictionary We'll introduce messages for 'rebase' soon, so we introduce a way to select a message based on the action being performed.

rebase: extract rebaseset and destination computation in a function The whole rebase function is gargantuan and this computation is almost 100 lines long. We extract it in a dedicated function as it is independent from the rest of the rebase code. Having it in its own function will make it easier to rework the default destination logic.

tests: have a more elaborated test for _destrebase The previous repo had a single lineage and a single head, pretty boring for rebase related testing.

histedit: break _histedit function into smaller pieces We add _getgoal, _validateargs. This is a part of bigger effort to refactor histedit. Initial steps are to break _histedit function into smaller pieces which will supposedly be more understandable. After this is done, I will have a better understanding of how histedit works and apply that to fix issue4800.

histedit: break _histedit function into smaller pieces (add _continueaction) This is a part of bigger effort to refactor histedit. Initial steps are to break _histedit function into smaller pieces which will supposedly be more understandable. After this is done, I will have a better understanding of how histedit works and apply that to fix issue4800.

histedit: break _histedit function into smaller pieces (add _newaction) This is a part of bigger effort to refactor histedit. Initial steps are to break _histedit function into smaller pieces which will supposedly be more understandable. After this is done, I will have a better understanding of how histedit works and apply that to fix issue4800.

histedit: break _histedit function into smaller pieces (add _editplanaction) This is a part of bigger effort to refactor histedit. Initial steps are to break _histedit function into smaller pieces which will supposedly be more understandable. After this is done, I will have a better understanding of how histedit works and apply that to fix issue4800.

histedit: break _histedit function into smaller pieces (add _abortaction) This is a part of bigger effort to refactor histedit. Initial steps are to break _histedit function into smaller pieces which will supposedly be more understandable. After this is done, I will have a better understanding of how histedit works and apply that to fix issue4800.

automv: new experimental extension Automatically detect moves and record them at commit time. This extension was originally developed at https://bitbucket.org/facebook/hg-experimental

match: override 'visitdir' in subdirmatcher The manifest.manifest class has a _treeinmem member than one can manually set to True to test that the treemanifest class works as a drop-in replacement for manifestdict (which is mostly a requirement for treemanifest repos to work). However, it doesn't quite work at the moment. These tests fail: test-largefiles-misc.t test-rebase-newancestor.t test-subrepo.t test-subrepo-deep-nested-change.t test-subrepo-recursion.t All but test-rebase-newancestor.t fail because they trigger calls to subdirmatcher.visitdir(), which tries to access a _excluderoots field that does not exist on the subdirmatcher. Let's fix that by overriding visitdir() in a similar way to how matchfn is overridden, i.e. by prepending the directory before calling the superclass method.

run-tests: remove useless "failed" flag from addOutputMismatch() It never be set to True.

tests: load json with no fallback The json module should be available in Python 2.6 or newer.

config: drop progress extension from sample hgrc as it is in core now

shelve: suggest the correct tool to continue (not unshelve) Suggest committing (or whatever the current activity is), via wrongtooltocontinue which uses howtocontinue.

histedit: suggest the correct tool to continue (not histedit) Suggest committing (or whatever the current activity is), via wrongtooltocontinue which uses howtocontinue.

rebase: suggest the correct tool to continue (not rebase) Suggest committing (or whatever the current activity is), via wrongtooltocontinue which uses howtocontinue.

graft: suggest the correct tool to continue (not graft) Add test coverage for graft --continue without starting. Suggest committing (or whatever the current activity is), via wrongtooltocontinue which uses howtocontinue.

cmdutil: provide a way to report how to continue checkafterresolved allows Mercurial to suggest what command to use next. If users try to continue the wrong command, there wasn't a good way for the command to suggest what to do next. Split checkmdutil into howtocontinue and checkafterresolved. Introduce wrongtooltocontinue which handles raising an Abort with the hint from howtocontinue.

hg: make cachedlocalrepo cache appropriate repoview object Before this patch, 'cachedlocalrepo' always caches "visible" repoview object, because 'cachedlocalrepo' uses "visible" repoview returned by 'hg.repository()' without any additional processing. If the client of 'cachedlocalrepo' wants "served" repoview, some objects to be cached are discarded unintentionally. 1. 'cachedlocalrepo' newly caches "visible" repoview object (call it VIEW1) 2. 'cachedlocalrepo' returns VIEW1 to the client of it at 'fetch()' 3. the client gets "served" repoview object by 'filtered("served")' on VIEW1 (call this "served" repoview VIEW2) 4. accessing to 'repo.changelog' implies: - instantiation of changelog via 'localrepository.changelog' - instantiation of "filtered changelog" via 'repoview.changelog' 5. "filtered changelog" above is cached in VIEW2 6. VIEW2 is discarded after processing, because there is no reference to it 7. 'cachedlocalrepo' returns VIEW1 cached at (1) above to the client at next 'fetch()' 8. 'filtered("served")' on VIEW1 at the client side creates new "served" repoview again, because VIEW1 is "visible" (call this new "served" repoview VIEW3) 9. accessing to 'repo.changelog' implies instantiation of filtered changelog again, because "filtered changelog" is cached in VIEW2 at (5), but not in VIEW3 currently used 10. (go to (7) above) As described above, "served" repoview object and "filtered changelog" cached in it are discarded always, even if the repository itself hasn't been changed since last access. For example, in the case of 'hgweb_mod.hgweb', "newly caching" occurs, when: - all cached objects are already assigned to another threads (in this case, repoview is created in 'cachedlocalrepo.copy()') - or, stat of '00changelog.i' is changed from last access (in this case, repoview is created in 'cachedlocalrepo.fetch()') once changes are pushed via HTTP, this always occurs. The root cause of this inefficiency is that 'cachedlocalrepo' always caches "visible" repoview object, even if the client of it wants another view. To make 'cachedlocalrepo' cache appropriate repoview object, this patch adds additional filtering on the repo object returned by 'hg.repository()'. It is assumed that initial repoview object should be already filtered by expected view. After this patch: - 'filtered("served")' on VIEW1 at (3)/(7) above returns VIEW1 itself, because VIEW1 is now "served", and - VIEW2 and VIEW3 equal VIEW1 - therefore, "filtered changelog" is cached in VIEW1, and reused intentionally

rebase: perform update through the 'update' command The update logic have grow more and more complicated over time (eg bookmark movement, new destination logic, warning on other head, etc). The rebase extension was reimplementing its own basic version of update to be used by 'hg pull --rebase'. We remove the custom code and use a combination of higher level functions. A test is added to check that the update is properly warning about other branch heads.

rebase: 'hg pull --rebase' now update only if there was nothing to rebase I recently discovered that 'hg pull --rebase' was also running an update. And it was running it in all cases as long as the update would move the working copy somewhere else... This felt wrong and it actually is. This 'update' call is introduced in 92455c1d6f83. In that commit the intent is very clear. The update should happen only when there was nothing to rebase. The implementation did not check if a rebase was performed because, at that time, rebase would always leave you on the top most changeset. Being on that top most changeset result in a no-op update and the step was skipped. However 9c78ed396075f changed rebase behavior to preserve the working copy parent, so if we are not on a head at pull time, the code performs both a rebase and an update. This changeset introduce a test for this case and restore the intended behavior. There are other issues with this custom update code but they will be addressed in later changeset (eg: own destination logic, lack of heads warning). I'm not super happy with the explicitly comparison 'rebase(...) == 1' but a later series will have a cleaner way to handle it anyway (while making 'rebase' pick its default destination like 'merge').

filectx: replace use of _filerev with _filenode _filerev depends on the filelog implementation using revlogs and linkrevs. Alternative implementations, like remotefilelog, do not have rev numbers, so this call fails. Replacing it with _filenode means it doesn't rely on rev numbers, and doesn't cost anything extra, since _filerev is using _filenode under the hood anyway.

verify: extract "manifest" constant into variable The "manifest" label that's used in error messages will instead be the directory path for subdirectory manifests (not the root manifest), so let's extract the constant to a variable already to make future patches simpler.

verify: use similar language for missing manifest and file revisions When a changeset refers to a manifest revision that's not found in the manifest log, we say "changeset refers to missing revision X", but when a manifest refers to file revision that's not found in the filelog, we say "X in manifests not found". The language used for missing manifest revisions seems clearer, so let's use that for missing filelog revisions too.

verify: include "manifest" prefix in a few more places We include the "manifest" prefix on most other errors, so it seems consistent to add them to the remaining messages too. Also, having the "manifest" prefix will be more consistent with having the directory prefix there when we add support for treemanifests. With the "manifest" at the beginning, let's remove the now-redundant "manifest" in the message itself.

verify: drop unnecessary check for nullid In eb914541a950 (verify: filter messages about missing null manifests (issue2900), 2011-07-13), we started ignoring nullid in the list of manifest nodeids to check. Then, in b32a30da608d (verify: do not choke on valid changelog without manifest, 2012-08-21), we stopped adding nullid to the list to start with. So let's drop the left-over check now.

verify: move cross-checking of changeset/manifest out of _crosscheckfiles() Reasons: * _crosscheckfiles(), as the name suggests, is about checking that the set of files files mentioned in changesets match the set of files mentioned in the manifests. * The "checking" in _crosscheckfiles() looked rather strange, as it just emitted an error for *every* entry in mflinkrevs. The reason was that these were the entries remaining after the call to _verifymanifest(). Moving all the processing of mflinkrevs into _verifymanifest() makes it much clearer that it's the remaining entries that are a problem. Functional change: progress is no longer reported for "crosschecking" of missing manifest entries. Since the crosschecking phase takes a tiny fraction of the verification, I don't think this is a problem. Also, any reports of "changeset refers to unknown manifest" will now come before "crosschecking files in changesets and manifests".

tests: add tests for missing revlogs and revlog entries The verify code is pretty poorly tested. It's easy to test missing revlogs and missing revlog entries, so let's add tests for that. Also add some more tests corrupting each type of revlog, so we test the messages presented when reading a revision fails. The pure and native implementations produce different error messages, so we have to use (glob) in the tests.

hook: don't crash on syntax errors in python hooks We had some real-world cases where syntax errors in Python hooks would crash the whole process and leave it in an indeterminate state. Handle those better.

hook: for python hook exceptions, add note to run with --traceback Just like with ImportErrors, it isn't obvious that --traceback will produce helpful debugging output here.

hook: add tests for failing post- python hooks I couldn't find any tests for this.

hook: even fewer parentheses for load errors Missed this one.

destutil: document various failure cases We document what various conditional branch mean and clarify that they are exclusive (since they all end up in with exception raised).