memctx: allow the metadataonlyctx thats reusing the manifest node When we have a lot of files writing a new manifest revision can be expensive. This commit adds a possibility for memctx to reuse a manifest from a different commit. This can be beneficial for commands that are creating metadata changes without any actual files changed like "hg metaedit" in evolve extension. I will send the change for evolve that leverages this once this is accepted.

localrepo: make it possible to reuse manifest when commiting context This makes the commit function understand the context that's reusing manifest.

manifest: expose the parents() method

httppeer: assign Vary request header last In preparation for adding another value to it in a subsequent patch. While I was here, I added some empty lines because walls of text are hard to read.

wireproto: only advertise HTTP-specific capabilities to HTTP peers (BC) Previously, the capabilities list was protocol agnostic and we advertised the same capabilities list to all clients, regardless of transport protocol. A few capabilities are specific to HTTP. I see no good reason why we should advertise them to SSH clients. So this patch limits their advertisement to HTTP clients. This patch is BC, but SSH clients shouldn't be using the removed capabilities so there should be no impact.

protocol: declare transport protocol name We add an attribute to the HTTP and SSH protocol implementations identifying the transport so future patches can conditionally expose capabilities on a per-transport basis.

bdiff: early pruning of common prefix before doing expensive computations It seems quite common that files don't change completely. New lines are often pretty much appended, and modifications will often only change a small section of the file which on average will be in the middle. There can thus be a big win by pruning a common prefix before starting the more expensive search for longest common substrings. Worst case, it will scan through a long sequence of similar bytes without encountering a newline. Splitlines will then have to do the same again ... twice for each side. If similar lines are found, splitlines will save the double iteration and hashing of the lines ... plus there will be less lines to find common substrings in. This change might in some cases make the algorith pick shorter or less optimal common substrings. We can't have the cake and eat it. This make hg --time bundle --base null -r 4.0 go from 14.5 to 15 s - a 3% increase. On mozilla-unified: perfbdiff -m 3041e4d59df2 ! wall 0.053088 comb 0.060000 user 0.060000 sys 0.000000 (best of 100) to ! wall 0.024618 comb 0.020000 user 0.020000 sys 0.000000 (best of 116) perfbdiff 0e9928989e9c --alldata --count 10 ! wall 0.702075 comb 0.700000 user 0.700000 sys 0.000000 (best of 15) to ! wall 0.579235 comb 0.580000 user 0.580000 sys 0.000000 (best of 18)

formatter: add overview of API and example as doctest

ui: factor out ui.load() to create a ui without loading configs (API) This allows us to write doctests depending on a ui object, but not on global configs. ui.load() is a class method so we can do wsgiui.load(). All ui() calls but for doctests are replaced with ui.load(). Some of them could be changed to not load configs later.

check-code: add a rule to forbid "cp -r" See the commit message of the previous patch for the reason. In short, according to the current POSIX standard, "-r" is "removed", and "-R" is the current standard way to do "copy file hierarchies".

tests: replace "cp -r" with "cp -R" The POSIX documentation about "cp" [1] says: .... RATIONALE .... Earlier versions of this standard included support for the -r option to copy file hierarchies. The -r option is historical practice on BSD and BSD-derived systems. This option is no longer specified by POSIX.1-2008 but may be present in some implementations. The -R option was added as a close synonym to the -r option, selected for consistency with all other options in this volume of POSIX.1-2008 that do recursive directory descent. The difference between -R and the removed -r option is in the treatment by cp of file types other than regular and directory. It was implementation-defined how the - option treated special files to allow both historical implementations and those that chose to support -r with the same abilities as -R defined by this volume of POSIX.1-2008. The original -r flag, for historic reasons, did not handle special files any differently from regular files, but always read the file and copied its contents. This had obvious problems in the presence of special file types; for example, character devices, FIFOs, and sockets. .... .... Issue 6 The -r option is marked obsolescent. .... Issue 7 .... The obsolescent -r option is removed. .... (No "Issue 8" yet) Therefore it's clear that "cp -R" is strictly better than "cp -r". The issue was discovered when running tests on OS X after 0d87b1caed92. [1]: pubs.opengroup.org/onlinepubs/9699919799/utilities/cp.html

posix: give the cached symlink a real target The NamedTemporaryFile file is cleared up so checklink ends up as a dangling symlink, causing cp -r in tests to complain on both Solaris and OS X. Use a permanent file instead when there is a .hg/cache directory.

shelve: move patch extension to a string constant We are using 'name + ".patch"' pattern throughout the shelve code to identify the existence of a shelve with a particular name. In two cases however we use 'name + ".hg"' instead. This commit makes 'patch' be used in all places and "emphasizes" it by moving 'patch' to live in a constant. Also, this allows to extract file name without extension like this: f[:-(1 + len(patchextension))] instead of: f[:-6] which is good IMO. This is a first patch from this initial "obsshelve" series. This series does not include tests, although locally I have all of test-shelve.t ported to test obs-shelve as well. I will send tests later as a separate series.

merge with stable

shelve: fix use of unexpected working dirs in test-shelve.t Fixing some clowniness where we created ~four levels of nested repos and once (my test case :( ) did not even cd into a created repo.

crecord: change the verb according to the operation This will make crecord consistent with record when being used in the revert situation. It will say "Select hunks to revert / discard" accordingly. This should make the revert crecord interface less confusing.

crecord: change help text for the space key dynamically A follow-up of the previous patch, to make the text simple and clear about whether it's to "select" or "deselect".

crecord: rewrite status line text (BC) Previously, we display fixed text in the 2-line status header. Now we want to customize some word there to make the "revert" action clear. However, if we simply replace the verb using '%s' like this: "SELECT CHUNKS: (j/k/up/dn/pgup/pgdn) move cursor; " "(space/A) toggle hunk/all; (e)dit hunk;"), " (f)old/unfold; (c)onfirm %s; (q)uit; (?) help " % verb "| [X]=hunk %s **=folded, toggle [a]mend mode" % verb It could cause trouble for i18n - some languages may expect things like "%(verb) confirm", for example. Therefore, this patch chooses to break the hard-coded 2-line sentences into "segment"s which could be translated (and replaced) separately. With the clean-up, I'm also changing the content being displayed, to make it cleaner and more friendly to (new) users, namely: - Replace "SELECT CHUNKS" to "Select hunks to record". Because: - To eliminate "hunk" / "chunk" inconsistency. - "record" is used in the "text" UI. Do not use "apply", to make it consistent. - To make it clear you are choosing what to record, not revert, or discard etc. This is probably the most important information the user should know. So let's put it first. - "to record" could be replaced to others depending on the operation. The follow-up patches will address them. - Move "[x]" and "**" legends first to explain the current interface. New users should understand what the legends mean, followed by what they can do in the interface. - Replace "j/k/up/dn/pgup/pgdn" with "arrow keys". Because: - "arrow keys" is more friendly to new users. - Mentioning "j/k" first may make emacs users angry. We should stay neutral about editors. - "pgup/pgdn" actually don't work very well. For example, within a hunk of 100-line insertion, "pgdn" just moves one single line. - Left/Right arrow keys are useful for movement and discovery of "expanding" a block. - Replace "fold/unfold" with "collapse/expand", "fold" is well known as a different meaning in histedit and evolve. - Replace "(space/A) toggle hunk/all" to "space: select". Because: - "A: toggle all" is not that useful - It's unclear how "hunk" could be "toggled" to a dumb user. Let's make it clear it's all about "select". - A follow-up will make it use "unselect" when we know the current item is selected. - Remove "(f)old". Use arrow keys instead. - Remove "toggle [a]mend mode". It's just confusing and could be useless. - Remove "(e)dit hunk". It's powerful but not friendly to new users. - Replace "(q)uit" to "q: abort" to make it clear you will lose changes. The result looks like the following in a 73-char-width terminal: Select hunks to record - [x]=selected **=collapsed c: confirm q: abort arrow keys: move/expand/collapse space: select ?: help If the terminal is 132-char wide, the text could fit in a single line.

crecord: make _getstatuslines update numstatuslines We are going to make the text in the status window dynamically generated, so its size would be dynamic. Change getstatuslines to update "numstatuslines" automatically. Fix an issue where "numstatuslines" being 1 makes the chunkpad disappear.

crecord: move status window text calculation to a separate method We will do some changes there in the next patches. The new method would also be the "source of truth" of the content of the status window (so if the status window needs more than 2 lines, it would be calculated from the new method). Also, moved "statuswin.refresh" to make the code compact and easier to read.

revlog: ensure that flags do not overflow 2 bytes This patch adds a line that ensures we are not setting by mistake a set of flags overlfowing the 2 bytes they are allocated. Given the way the data is packed in the revlog header, overflowing 2 bytes will result in setting a wrong offset.

merge with stable

debugcommands: sort command order The diff is a bit large but it is straight code moving without any logical modifications.

tests: add test that @commands in debugcommands.py are sorted I felt like inline Python in test-check-code.t was the most appropriate place for this, as other linters in contrib/ seem to be source file agnostic. The test currently fails.

fsmonitor: be robust in the face of bad state fsmonitor could write out bad state if interrupted part way through, and would then crash when it tried to read it back in. Make both sides of the operation more robust - reading state should fail cleanly, and we can use atomictemp to write out cleanly as the file is small. Between the two, we shouldn't crash with an IndexError any more.

merge: use original file extension for temporary files Some merge tools (like Araxis?) can pick merge mode based on the file extension. That didn't work well when temporary files were given random suffixes. It seems to work better when the random part is before the extension. As usual, when using $output, $local will have the .orig extension. That could perhaps be the subject of another change another day.

ui: use try..finally in configoverride @contextmanager almost always have their "yield" inside a try..finally block. This is because if the calling code inside the activated context manager raises, the code after the "yield" won't get executed. A "finally" block, however, will get executed in this scenario.

util: limit output chunk size in zlib decompression This is essentially a port of 65bd4b8e48bd, which was inadvertently dropped by 8cd7d0fefd30.

crecord: filter text via i18n There are some text in the user interface that are not filtered by i18n. This patch adds the missing "_" call. So the text could be translated.

revert: pass operation to crecord So crecord would know what to display

crecord: add an "operation" field The field would provide extra information to help us to make the curses UI text less confusing.

revert: prompt before removing files in interactive mode Prior to this change, files to be removed (i.e. files added since the revision to revert to) were unconditionally removed despite the interactive mode. Now prompt before actually removing the files, as this is done for other actions (e.g. forget).

revert: indicate the default choice when prompting to forget files

style: avoid an unnecessary line split

debugcommands: move 'debugdeltachain' in the new module

debugcommands: move 'debugindex' and 'debugindexdot' in the new module

debugcommands: move 'debugignore' in the new module

debugcommands: move 'debuggetbundle' in the new module

debugcommands: move 'debugfsinfo' in the new module

debugcommands: move 'debugfileset' in the new module

censor: flag internal documentation

shelve: make --keep option survive user intervention (issue5431) Currently if user runs 'hg unshelve --keep' and merge conflicts occur, the information about --keep provided by user is lost and shelf is deleted after 'hg unshelve --continue'. This is obviously not desired, so this patch fixes it.

worker: use os._exit for posix worker in all cases Like commandserver, the worker should never run other resource cleanup logic. Previously this is not true for workers if they have exceptions other than KeyboardInterrupt. This actually caused a real-world deadlock with remotefilelog: 1. remotefilelog/fileserverclient creates a sshpeer. pipei/o/e get created. 2. worker inherits that sshpeer's pipei/o/e. 3. worker runs sshpeer.cleanup (only happens without os._exit) 4. worker closes pipeo/i, which will normally make the sshpeer read EOF from its stdin and exit. But the master process still have pipeo, so no EOF. 5. worker reads pipee (stderr of sshpeer), which never completes because the ssh process does not exit, does not close its stderr. 6. master waits for all workers, which never completes because they never complete sshpeer.cleanup. This could also be addressed by closing these fds after fork, which is not easy because Python 2.x does not have an official "afterfork" hook. Hacking os.fork is also ugly. Besides, sshpeer is probably not the only troublemarker. The patch changes _posixworker so all its code paths will use os._exit to avoid running unwanted resource clean-ups.

dispatch: move part of callcatch to scmutil Per discussion at 39149b6036e6 [1], we need "callcatch" in worker.py. Move it to scmutil.py to avoid cycles. Note that dispatch's callcatch handles some additional high-level exceptions related to config parsing, and commands. Moving them to scmutil will make scmutil depend on "commands" or require "_formatparse" and "_getsimilar" (and "difflib") to be moved as well. In the worker use-case, it is forked when config and commands are fully loaded. So it should not care about those exceptions. [1]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-August/087116.html

py3: use pycompat.getcwd() instead of os.getcwd() We have pycompat.getcwd() which returns bytes path on Python 3. This patch changes most of the occurences of the os.getcwd() with pycompat one.

debugcommands: move 'debugextensions' to the new module

debugcommands: move 'debugdiscovery' in the module And a lot of imports with it.

debugcommands: move 'debugdate' in the new module

debugcommands: move 'debugrevlogopts' into the new module This move contains the first reference to debugrevlogopts in debugcommands.py. We'll eventually want to move that over. We hold off for now because it would introduce a module import cycle.

debugcommands: move 'debugdag' into the new module

chgserver: make it a core module and drop extension flags It was an extension just because there were several dependency cycles I needed to address. I don't add 'chgserver' to extensions._builtin since chgserver is considered an internal extension so nobody should enable it by their config.

chgserver: delay importing commands and dispatch modules This is a workaround for future import cycle: dispatch -> commands -> server -> chgserver -> commands. Some of the problems can be fixed later on pager and chg refactoring.

chgserver: drop CHGINTERNALMARK by chgunixservice() Prepares for the removal of uisetup(). We just need to do that at the start of the chg server, so chgunixservice() should be fine.

server: add public function to select either cmdserver or hgweb

server: move service factory from hgweb

hgweb: extract app factory I'll move createservice() to the server module, but createapp() seems good to remain in the hgweb module because of its dependency on hgweb/hgwebdir_mod.

server: move service table and factory from commandserver This is necessary to solve future dependency cycle between commandserver.py and chgserver.py. 'cmd' prefix is added to table and function names to avoid conflicts with hgweb.

server: move cmdutil.service() to new module (API) And call it runservice() because I'll soon add createservice(). The main reason I'm going to introduce the 'server' module is to solve future dependency cycle between chgserver.py and commandserver.py. The 'server' module sits at the same layer as the cmdutil. I believe it's generally good to get rid of things from the big cmdutil module.

debugcommands: move 'debugcomplete' in the new module

debugcommands: move 'debugcommands' in the new module The commit message isn't an illusion. There is a "debugcommands" module and command.