mpatch: reflow two oddly formatted else blocks with clang-format Differential Revision: https://phab.mercurial-scm.org/D1029

mpatch: re-wrap wide line with clang-format Differential Revision: https://phab.mercurial-scm.org/D1027

bdiff: remove trailing newlines Differential Revision: https://phab.mercurial-scm.org/D1009

bdiff: rewrap function prototypes per clang-format Differential Revision: https://phab.mercurial-scm.org/D1008

bdiff: re-wrap lines per clang-format A few too-wide lines corrected, and some places where clang-format prefers to wrap after the binary operator instead of before. I don't feel strongly, so I'm leaving the auto-format result as "after the binary operator". Differential Revision: https://phab.mercurial-scm.org/D1007

bdiff: remove extra space after * per clang-format Differential Revision: https://phab.mercurial-scm.org/D1006

bdiff: fix misplaced comma in macro definition with clang-format Differential Revision: https://phab.mercurial-scm.org/D1005

bdiff: format header file with clang-format Differential Revision: https://phab.mercurial-scm.org/D1004

bdiff: sort includes using clang-format Differential Revision: https://phab.mercurial-scm.org/D1003

build: "make deb" failed when the base path contained spaces With these changes, all the commands triggered by "make deb" use proper quoting and succeed even when invoked from a directory containing spaces.

build: chg build was failing when the base directory contained spaces

build: initial version detection by make deb/rpm was missing quoting

build: make install in "/doc" failed if the destination dir contained spaces This and the following commits try to add the necessary quoting in the build scripts to make the process more robust. The target for now is rendering "make deb" successful even when the base directory contains spaces (eg. "/opt/mercu rial"). The build process should succeed without scattering files in spurious directories (eg.: "/opt/mercu/usr/bin/hg").

strip: take branch into account when selecting update target (issue5540) Test contributed by Matt Harbison Keep the same behavior in most cases (i.e. first parent of the first root of stripped changsets), but if the branch differs from wdir's, try to find another parent of stripped commits that is on the same branch.

scmutil: factor out building of transaction summary callback In registersummarycallback(), we extra generic bits of the existing "reportsummary" function into a decorator which will be used in forthcoming changesets to add new summary callbacks.

scmutil: factor out transaction name lookup in registersummarycallback() Add an inner txmatch function in registersummarycallback() factoring out the logic to determine if the transaction matches a particular sources set. We'll reuse this function to add some new report logic in the new changeset.

configitems: register the annotate diff options

configitems: register the 'convert.cvsps.logencoding' config

configitems: register the 'ui.interface.chunkselector' config

configitems: register the 'experimental.archivemetatemplate' config

configitems: register the 'rebase.singletransaction' config

configitems: register the 'server.bundle*' family of config All these config use the same function specifying a default value. We need to register them all at the same time.

configitems: register the 'web.logourl' config

configitems: register the 'web.logoimg' config

configitems: register the 'web.labels' config

configitems: register the 'web.hidden' config

configitems: register the 'web.guessmime' config

configitems: register the 'web.deny_push' config

configitems: register the 'web.contact' config

configitems: register the 'web.cache' config

configitems: register the 'web.allowzip' config

configitems: register the 'web.allow_push' config

configitems: register the 'web.allowpull' config

configitems: register the 'web.allowgz' config

configitems: register the 'web.allowbz2' config

configitems: register the 'email.reply-to' config

configitems: register the 'email.cc' config

configitems: register the 'email.bcc' config

configitems: register the 'logtoprocess.uiblocked' config

configitems: register the 'logtoprocess.develwarn' config

configitems: register the 'logtoprocess.command' config

configitems: register the 'logtoprocess.commandfinish' config

configitems: register the 'logtoprocess.commandexception' config

configitems: register the 'pager.pager' config

configitems: register the 'web.maxchanges' config

configitems: register the 'web.maxfiles' config

configitems: register the 'web.maxshortchanges' config

configitems: register the 'web.motd' config

configitems: register the 'web.name' config

configitems: register the 'web.push_ssl' config

configitems: register the 'web.view' config

configitems: fix registration for 'blackbox.track' config Default mutable values could be problematic. Use a lambda that returns the value instead. Thanks to martin for catching this bug.

status: copy-edit and improve --terse and --verbose help

templater: store revisions as ints so min/max won't compare them as strings Because a template value has no explicit type (like ancient PHP), ifcontains() has to coerce the type of the needle. Before, it was always converted to a string, which meant any container type should be a list/dict of strings. This no longer works since we've introduced min/max functions. In order to work around the untyped nature of templater, this patch adds a type specifier to hybrid dict/list. It isn't named as "valuetype" since the _hybrid class can also wrap a dict.

fsmonitor: update to match new dirstate refactor The dirstate was refactored so dirstate._map is now at dirstate._map._map. Same for _copymap, is not _map.copymap. It seems none of the mercurial tests cover this stuff, but it was caught by our Facebook extension tests. Differential Revision: https://phab.mercurial-scm.org/D945

doc: rename README to README.rst Many hosting services consider README without extension as plain text. By using .rst extension, we bring better formatting on many services (e.g. bitbucket).

test: use current user in the multi-user amend scenario The obsfate template keyword that will be introduced in a later patch distinguish between current user and other users when displaying the obsfate users. Update the test before for clarity.

test: cleanup verbosity variant from fatelog test Unlike the '{obsfate}' keyword defined in 'hg-evolve', the definition of hg fatelog in test-obsmarker-template.t is not verbosity dependent. Remove useless call to fatelog -v and fatelog -q.

test: rename obsfate template alias We will introduce an obsfate templatekw in the next patch that will conflict with the obsfate template in the test file, rename the one in test file to avoid this collision.

obsolete: add a devel.user.obsmarker In tests, we often want to use a different username in obs-markers without changing the default user. Add a 'devel.user.obsmarker'.

strip: factor out update target selection The same algorithm was used in two places: one to find out which commit shall become the parent of wdir, and the other to prepare the wdir when keeping changes. Factoring it out prevents inconsistent changes in either occurrence.

contrib: add check-code rule banning use of readlink readlink doesn't always exist, such as on Solaris-derived platforms. Differential Revision: https://phab.mercurial-scm.org/D1000

tests: use readlink.py instead of readlink The latter doesn't always exist, such as on Solaris-derived platforms. Differential Revision: https://phab.mercurial-scm.org/D999

tests: use readlink.py instead of readlink The latter doesn't always exist, such as on Solaris-derived platforms. Differential Revision: https://phab.mercurial-scm.org/D998

tests: use readlink.py instead of readlink The latter doesn't always exist, such as on Solaris-derived platforms. Differential Revision: https://phab.mercurial-scm.org/D997

tests: use readlink.py instead of readlink The latter doesn't always exist, such as on Solaris-derived platforms. Differential Revision: https://phab.mercurial-scm.org/D996

ui: recommend tweakdefaults in the default hgrc template Were I only slightly bolder, I'd make it on by default in the suggested user hgrc. Differential Revision: https://phab.mercurial-scm.org/D1001

tests: optimize test-copytrace-heuristics.t test-copytrace-heuristics.t tests the heuristics algorithm of copytracing. The test has a pattern of making a server repo first and then cloning into a local repo for each test to have public and draft commits. This is done because if we have all draft commits and we have very less commits, heuristics will fallback to full copytracing as that will be fast. To avoid creating the server repo, we set the commit limit to -1 so that everytime we perform the heuristics algorithm even when having full drafts and tweak the config setting when we need to test the ability to fallback to full copytracing. This optimizes this test by 3 seconds. Before: real 0m41.503s user 0m36.068s sys 0m3.900s After: real 0m38.826s user 0m33.884s sys 0m3.396s Differential Revision: https://phab.mercurial-scm.org/D991

patch: do not cache translated messages (API) Previously the code caches `i18n._` results in module variables. That causes issues after an encoding change. Instead of invalidating them manually, we now just recalculate the translated messages every time `filterpatch` gets called. This makes test-commit-interactive.t pass regardless of whether chg or demandimport is used or not. .. api: `patch.messages` now lives in `patch.getmessages()`. Extensions adding new messages should now wrap the `patch.getmessages` method instead of changing `patch.messages` directly. Differential Revision: https://phab.mercurial-scm.org/D959

fsmonitor: add new watchman notifications to fsmonitor extension The fsmonitor extension currently sends state-enter and state-leave notifications to watchman on the update operation. This commit creates additional notifications for the following events : - transaction creation and commit/abort. A state-enter notification will be sent when a transaction is created. It will provide the working copy parent's hash. A state-leave notification will be sent when the transaction is committed or aborted. It will provide the working copy parent's hash. - calls to set-parent will cause state-enter and state-leave notifications to be sent. The state-enter notification will be sent prior to the set-parent operation and the working copy parent's hash will be provided at this time. The state-leave notification will be sent after the set-parent operation completes providing the working copy parents hash. Test Plan: tested on dev server to check that necessary notifications were sent/received Differential Revision: https://phab.mercurial-scm.org/D989

fsmonitor: change the distance calculation Change the distance calculation in the fsmonitor extension. It is done in a method since anticipated changes will need to use this logic as well. Test Plan: Tested on development server. Differential Revision: https://phab.mercurial-scm.org/D988

configitems: update default value of 'phases.new-commit' Now that mq is cleaned up, we can rely on the config register for the default value. We switch the default from the integer to human readable value to help with future automatic documentation that could be generated from the config register.

mq: use the newcommitphase utility We use the official API instead of doing the operation manually. This will help storing the default value for phases properly in the next changeset.

patch: rename "header" variable into "hdr" in diff() The "header" variable was hiding the eponymous class, hence preventing its usage.

merge: improve comments in mergestate._makerecords Differential Revision: https://phab.mercurial-scm.org/D955

commands: tidy up merge state decoding in resolve Make the mapping from merge state to label and display key explicit, and move construction of the dict out of the loop. Differential Revision: https://phab.mercurial-scm.org/D861

tests: add test for path conflicts during merge Differential Revision: https://phab.mercurial-scm.org/D787

tests: add test for path conflicts during update Differential Revision: https://phab.mercurial-scm.org/D786

context: also consider path conflicts when clearing unknown files When clearing unknown files to remove path conflicts, also delete files that conflict with the target file's path. Differential Revision: https://phab.mercurial-scm.org/D785

merge: check for path conflicts when merging (issue5628) When merging, check for any path conflicts introduced by the manifest merge and rename the conflicting file to a safe name. Differential Revision: https://phab.mercurial-scm.org/D784

util: add safename function for generating safe names to rename to This function finds a name which does not clash with any other name in the manifest, and so can be used to safely rename a file. Differential Revision: https://phab.mercurial-scm.org/D783

merge: improve error messages for path conflicts during update Differential Revision: https://phab.mercurial-scm.org/D782

merge: check for path conflicts when updating (issue5628) When updating to a new revision, check for path conflicts caused by unknown files in the working directory, and handle these by backing up the file or directory and replacing it. Differential Revision: https://phab.mercurial-scm.org/D781

merge: rename conflicts to fileconflicts in _checkunknownfiles We will need to distinguish between file conflicts and path conflicts. Rename the conflicts variable so that it will be clearly distinct from pathconflicts, which will be introduced in a future commit. Differential Revision: https://phab.mercurial-scm.org/D780

merge: add _checkunknowndirs function for detecting path conflicts Add a new function which, given a file name, finds the shortest path for which there is a conflicting file or directory in the working directory. Differential Revision: https://phab.mercurial-scm.org/D779

merge: backup conflicting directories when getting files During batchget, if a target file conflicts with a directory, or if the directory a target file is in conflicts with a file, backup and remove the conflicting file or directory before performing the get. Differential Revision: https://phab.mercurial-scm.org/D778

merge: add merge action 'pr' to rename files during update Add a new merge action to handle a path conflict by renaming the conflicting file to a safe name. The rename is just to avoid problems on the filesystem. The conflict is still considered unresolved until the user marks the original path as resolved. Differential Revision: https://phab.mercurial-scm.org/D777

merge: add merge action 'p' to record path conflicts during update Add a new merge action to record path conflicts. A status message is printed, and the path conflict is added to the merge state. Differential Revision: https://phab.mercurial-scm.org/D776

commands: update the resolve command to handle path conflicts The resolve command must support displaying path conflicts and marking them as resolved or unresolved. Differential Revision: https://phab.mercurial-scm.org/D775

merge: add pathconflict merge state Path conflicts that occur during merges are represented by 'pu' (unresolved) and 'pr' (resolved) records in the merge state. These are stored on disk in 'P' records. Differential Revision: https://phab.mercurial-scm.org/D774

tests: add a test demonstrating basic path conflict failures Differential Revision: https://phab.mercurial-scm.org/D773

scmutil: handle conflicting files and dirs in origbackuppath When ui.origbackuppath is set, .orig files are stored outside of the working copy. However conflicts can occur when files or directories end up having the same name. These conflicts cause Mercurial to abort, even if they've been created as a result of different backups. Make sure we always replace files or directories in the origbackuppath if they conflict with another file or directory. Test Plan: Add new unit test for conflicting paths. Differential Revision: https://phab.mercurial-scm.org/D680

extdata: show debug message if external command exits with non-zero status This isn't fatal because it's quite common for grep to exit with 1. Thanks to Foozy for spotting this.

templatekw: allow accessing to nested namespace item by its template name Since we have the dot operator, it makes more sense to write {namespaces.tags % "{tag}"} instead of {namespaces.tags % "{name}"}

templatekw: get rid of temporary dicts from shownamespaces()

templatekw: rename peerpaths to peerurls per naming convention (BC) Since each element is called as "url", the template keyword should be named as "<whatever>urls". {peerurls} is now stabilized.

templatekw: make experimental {peerpaths} return a single-level dict (BC) This was planned as in c0d8de2724ce, "{peerpaths.default.pushurl} will be translated to peerpaths['default'].makemap()['pushurl'], which means {peerpaths} should be a single-level dict and sub-options should be makemap()-ed."

templatekw: fix scope of peerpath url bound to generator I had to explicitly bind 'd' to the generator. Otherwise, the last 'd' would be used.

templater: extend dot operator as a short for get(dict, key)

templater: add dot operator to easily access a sub item This and the next patch will allow us to access a deeply-nested item by foo.bar.baz syntax.

templater: wrap get/min/max result so map operation can apply to element See the test for usage example. wraphybridvalue() takes a key/value pair because a hybrid dict passes a key to its makemap() function. Since makemap() of showmanifest() doesn't need a key, it's set to None.

dispatch: move initialization of sys.std* files I'll add another Python 3 hack.

py3: work around the scope of exception variable in dispatch.run() https://stackoverflow.com/questions/29268892/

chg: just forward --time to command server Since we've removed the use of atexit in de5c9d0e02ea, --time just works.

windows: add an experimental option for long paths support This commit adds an experimental --long-paths-support flag to build_hgexe on Windows. It is off by default, but when supplied, causes setup.py to embed some XML into the generated hg.exe, which in turn tells Windows to allow this exe to use long paths (given that the appropriate registry setting is enabled as well). This was tested on Windows 10 14393 and 15063. This commit introduces a badly-named initialize_options function, but its name is dictated by distutils, rather than chosen. # no-check-commit

configitems: register the 'experimental.graphstyle.grandparent' config

configitems: register the 'experimental.graphstyle.missing' config

configitems: register the 'experimental.graphstyle.parent' config

configitems: register the 'devel.empty-changegroup' config

configitems: register the 'devel.cache-vfs' config

configitems: register the 'devel.warn-config-default' config

configitems: register the 'devel.warn-config' config

configitems: register 'merge.checkunknown' and 'merge.checkignored' They both use the same function defining a default, so we need to update them at the same time.

configitems: register the 'diff.*' config All the config were already using a unified function with a forced default, so, registering them all at once seems safe.

configitems: register the 'experimental.mmapindexthreshold' config

configitems: register the 'experimental.maxdeltachainspan' config

configitems: register the 'commands.show.aliasprefix' config

configitems: register the 'blackbox.track' config

copies: add docs for config `experimental.copytrace.sourcecommitlimit` This patch adds documentation for the config option. The config name does not convey much and hence documentation was required. Differential Revision: https://phab.mercurial-scm.org/D986

hgweb: extract function for loading style from request context Also make it work on Python 3. Differential Revision: https://phab.mercurial-scm.org/D970

request: coerce content-type to native str Again, required by WSGI. Differential Revision: https://phab.mercurial-scm.org/D969

request: use trivial iterator over dictionary keys Differential Revision: https://phab.mercurial-scm.org/D968

hgweb: when constructing or adding to a wsgi environ dict, use native strs That's what's required of us to work with the WSGI API on Python 3. Differential Revision: https://phab.mercurial-scm.org/D967

hgweb: produce native string for etag value Also use %d since we know mtime is numeric. Differential Revision: https://phab.mercurial-scm.org/D966

hgweb: in protocol adapter, look for bytes instances, not str Differential Revision: https://phab.mercurial-scm.org/D963

hgweb: in protocol adapter, avoid control reaching end of non-void function This greatly confounded some Python 3 porting work, once it was managing to get this far. Differential Revision: https://phab.mercurial-scm.org/D962

test-check-code: fail new commits which use 'atexit' instead of 'ui.atexit' Callbacks registered with 'atexit' are sometimes not called (like when hg aborts and calls os._exit). On the other hand, callbacks registered with 'ui.atexit' are called in most cases. Therefore, encouraging the use of 'ui.atexit' by failing the test 'test-check-code.t' appropriately. Test Plan: Ran the test 'test-check-code.t' after importing 'atexit' in one of the py files and confirmed that the test fails. Differential Revision: https://phab.mercurial-scm.org/D961

test-push-race: use 'ui.atexit' instead of python's 'atexit' Callbacks registered with 'atexit' are sometimes not called (like when a process is killed by the SIGTERM signal). Therefore, this commit replaces it with 'ui.atexit' which ensures that the callbacks are always called. This also makes the test compatible with chg. Test Plan: Ran the test 'test-push-race.t' with and without the '--chg' option. Differential Revision: https://phab.mercurial-scm.org/D957

py3: use '%d' for integers instead of '%s' Differential Revision: https://phab.mercurial-scm.org/D973

py3: fix keyword arguments handling in mq This patch fixes the handling of keyword arguments to functions on Python 3. On python3, the keys of keyword arguments need to str which is unicode. So any keyword argument will get will have str keys and any dictionary we pass as kwargs must have all the keys as str. This patch uses pycompat.(strkwargs|byteskwargs) to do so conversion between bytes keys and str keys and use r'' if there are very less uses and conversion can be prevented. Differential Revision: https://phab.mercurial-scm.org/D972

test-rebase-base: clarify it is about the "--base" flag It happened several times that people use `test-rebase-base.t` as a general purposed test file for rebase. But it is intended to be only related to the `--base` flag. This patch split, renamed the test, and added a note to clarify. Differential Revision: https://phab.mercurial-scm.org/D975

configitems: register the 'convert.p4.encoding' config

configitems: register the 'gpg.key' config

configitems: register the 'gpg.cmd' config

configitems: register the 'keywordset.svn' config

configitems: register the 'hgk.path' config

configitems: register the 'share.poolnaming' config

configitems: register the 'share.pool' config

configitems: register the 'shelve.maxbackups' config

configitems: register the 'pager.attend' config

configitems: register the 'perf.stub' config

configitems: register the 'commands.rebase.requiredest' config

configitems: register the 'experimental.rebaseskipobsolete' config

configitems: register the 'experimental.nonnormalparanoidcheck' config

configitems: register the 'experimental.allowdivergence' config

histedit: removing the experimental config 'histeditng' This config has been around for about 2 years now. So, it can be assumed to be stable. Therefore, I am removing the config. This also makes the test 'test-histedit-base.t' compatible with chg. Test Plan: - Ran the test 'test-histedit-base.t' with and without '--chg' option. - Ran all the other tests without '--chg' option. - Checked the output of 'hg help histedit'. Differential Revision: https://phab.mercurial-scm.org/D942

convert: fix the RevisionSpec import in the bzr module This import was failing (because its invalid) which was resulting in the following tests failing with chg running: - test-convert-bzr.t - test-convert-bzr-directories.t - test-convert-bzr-ghosts.t - test-convert-bzr-treeroot.t - test-convert-bzr-114.t - test-convert-bzr-merges.t This commit fixes the import which in turn fixes the tests. Test Plan: Ran the aforementioned tests with and without '--chg' option. Differential Revision: https://phab.mercurial-scm.org/D936

test-annotate: fix up expected output for pure This is just a side effect of surfacing the "revisions were skipped" state in the blame output. Differential Revision: https://phab.mercurial-scm.org/D956

style: never use a space before a colon or comma Differential Revision: https://phab.mercurial-scm.org/D954

eol: make [eol] config section sensitive for chg confighash The eol extension may mangle the [eol] config section and that means chg is unable to detect config file change (because it re-applies setconfig changes). This makes test-eol.t pass with chg. Differential Revision: https://phab.mercurial-scm.org/D917

test-alias: make it compatible with chg ad1bdea (dispatch: defer environment variable resolution in alias commands (BC), 2016-05-06) made environment variables lazily resolved. But after D805 (alias: make alias command lazily resolved 2017-09-23), alias resolution happened after uisetup, which breaks the test for chg. `uisetup` is known to behave different in chg. Let's verify the feature without using `*setup`. Differential Revision: https://phab.mercurial-scm.org/D912

serve: make tests compatible with chg chg only supports 'hg serve' when the options to the serve command follow the 'hg serve'. For example, 'hg -R <repo> serve ..' is unsupported. This leads to issues with chg running for the following tests: - test-bundle2-exchange.t - test-clone-uncompressed.t - test-hgweb-csp.t - test-http-bad-server.t - test-http-bundle1.t - test-http-protocol.t - test-http.t There was an effort made earlier to fix this issue for chg and the tests were fixed to confirm to the compatible pattern. But the new tests did not take care of the same and hence, fail. Hopefully, there will be continuous build setup for chg after all tests are made compatible with chg so that we can avoid such issues. Test Plan: Ran the aforementioned tests with and without '--chg' option. Differential Revision: https://phab.mercurial-scm.org/D946

ui: convert to/from Optional[bytes] to Optional[str] in password manager This password manager proxy is roughly the right-looking layer to convert between strings and bytes. Many of these arguments can be None, so we have a helper method to make the conversion preserve Nones without exploding. Differential Revision: https://phab.mercurial-scm.org/D886

configitems: register the 'debug.dirstate.delaywrite' config

configitems: register the 'merge.preferancestor' config

configitems: register the 'email.from' config

configitems: register the 'smtp.port' config

configitems: register the 'phases.new-commit' config

configitems: register the 'experimental.histeditng' config

configitems: register the 'experimental.histedit.autoverb' config

configitems: register the 'histedit.singletransaction' config

configitems: register the 'histedit.linelen' config

configitems: register the 'histedit.dropmissing' config

configitems: register the 'histedit.defaultrev' config

configitems: register the 'transplant.log' config

configitems: register the 'transplant.filter' config

urllibcompat: move some adapters from pycompat to urllibcompat These are all the httpserver and urllib.* aliases. They seem to make more sense in the slightly-more-specific urllibcompat package than the general-purpose pycompat. Differential Revision: https://phab.mercurial-scm.org/D935

cleanup: use urllibcompat for renamed methods on urllib request objects Differential Revision: https://phab.mercurial-scm.org/D891

urllibcompat: new library to help abstract out some python3 urllib2 stuff Doing a new file instead of pycompat because I'm starting to feel like pycompat is getting a little enormous in terms of scope. Differential Revision: https://phab.mercurial-scm.org/D890

test-devel-warnings: make the test compatible with chg The test fails when run with the "--chg" option. Therefore, this commit makes it compatible with chg. Test Plan: Ran the test "test-devel-warnings.t' with and without the "--chg" option Differential Revision: https://phab.mercurial-scm.org/D915

fsmonitor: use configitem We might as well get this out of the way. Differential Revision: https://phab.mercurial-scm.org/D893

fsmonitor: access copymap in new location fsmonitor has been busted since 0865d25e8a8a due to moving self._copymap. Fix it. Differential Revision: https://phab.mercurial-scm.org/D892

extdata: use subprocess so we don't have to chdir() manually

extdata: just use iterator to read lines one by one

extdata: ignore ambiguous identifier as well

templater: add experimental support for extdata This is minimal and non-controversial implementation of extdata() template function. Originally extdata sources were exposed to the keyword namespace, but I've changed it to a plain function for simplicity.

revset: add experimental support for extdata This is minimal and non-controversial implementation of extdata() revset. Originally extdata sources were exposed to the symbol namespace, but I've changed it to a plain function for simplicity.

extdata: add extdatasource reader This adds basic support for extdata, a way to add external data sources for revsets and templates. An extdata data source is simply a list of lines of the form: <revision identifier>[<space><freeform text>]\n An extdata source is configured thusly: [extdata] name = <a url or path> urls of the form shell: are launch shell commands to generate data. This patch is slightly modified by Yuya Nishihara as follows: - fix typo - remove unused function - remove future expansion point for parameter (which can be added later as the extdata revset/template are experimental) You can see the original patch at https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-September/088426.html

tests: change to parent directory before all "hg init" in test-log.t Add a "cd .." before the second "hg init" so that all repositories are on top level of test directory. Makes inspection of test directory easier.

config: add a missing preparewrite() call Thanks Yuya for pointing this out in D808. Differential Revision: https://phab.mercurial-scm.org/D924

test-strip: make test compatible with chg The test was using reposetup which had the logic for stripping commits. This leads to a situation where if the reposetup was called twice for an extension (which can happen with chg running), the stripped node would not be found the second time. Therefore, this commit changes the test to put the stripping logic inside commands instead of the reposetup. This ensures that the stripping logic is invoked only when the command is invoked and thus, avoids any problems. Test Plan: Ran the test 'test-strip.t' with and without the '--chg' option. Differential Revision: https://phab.mercurial-scm.org/D928

test-hook: make test compatible with chg The test uses the 'print' method instead of writing to stdout using 'ui.write' which leads to incompatibility with chg. This commit modifies the test to use 'ui' instead which fixes the problem. Test Plan: Ran the test 'test-hook.t' with and without '--chg' option. Differential Revision: https://phab.mercurial-scm.org/D927

test-setdiscovery: make test compatible with chg The test checks the output of the blackbox extension which will contain logs corresponding to chg in case chg is running. Therefore, this commit modifies the test to take chg into consideration while working with the blackbox extension. Test Plan: Ran the test 'test-setdiscovery.t' with and without the '--chg' option. Differential Revision: https://phab.mercurial-scm.org/D926

test-merge-subrepos: make test compatible with chg The test checks the output of '.hg/blackbox.log' which will contain logs corresponding to chg in case chg is running. Therefore, this commit modifies the test to take chg into consideration while checking the blackbox.log contents. Test Plan: Ran the test 'test-merge-subrepos.t' with and without the '--chg' option. Differential Revision: https://phab.mercurial-scm.org/D925

test-convert-cvs: make test compatible with chg The test uses the 'print' method instead of writing to stdout using 'ui.write' which leads to incompatibility with chg. This commit modifies the test to use the 'ui' object instead which fixes the problem. Test Plan: Ran the test 'test-convert-cvs.t' with and without '--chg' option. Differential Revision: https://phab.mercurial-scm.org/D923

test-basic: make test compatible with chg The error codes returned when writing to /dev/full are different after the first failure with and without '--chg' option. Therefore, this commit conditionally handles the error codes as appropriate. Test Plan: Ran the test 'test-basic.t' with and without '--chg' option. Differential Revision: https://phab.mercurial-scm.org/D922

zeroconf: do not crash if socket being read is closed by another thread In zeroconf/__init__.py, there is: server = Zeroconf.Zeroconf(ip) l = listener() Zeroconf.ServiceBrowser(server, "_hg._tcp.local.", l) time.sleep(1) server.close() `server.close()` closes the underlying socket while the `ServiceBrowser` may still have a background thread reading the socket. There could be a race condition where the reading thread reads the closed socket, resulting in EBADF crash. This patch catches the exception. This makes test-paths.t pass with chg. Differential Revision: https://phab.mercurial-scm.org/D919

test-revlog-mmapindex: make it compatible with chg The test misses an explicit flush(). Differential Revision: https://phab.mercurial-scm.org/D918

test-profile: gate chg-incompatible part with '#if chg' chg has a different extension loading logic, which affects the profiler extension test case. Gate the block with '#if chg' so the test passes with chg. Differential Revision: https://phab.mercurial-scm.org/D916

test-logtoprocess: make it compatible with chg chg runs more commands and outputs more lines. This patch matches them. Differential Revision: https://phab.mercurial-scm.org/D914

test-globalopts: make the test compatible with chg The test fails when run with the '--chg' option. Therefore, this commit modifies the test to make it compatible with chg. Test Plan: Ran 'test-globalopts.t' with and without the '--chg' option. Differential Revision: https://phab.mercurial-scm.org/D913

test-pager: make it compatible with chg chg's runpager implementation is different. It behaves differently for the "shell=False, command not found" case. Differential Revision: https://phab.mercurial-scm.org/D911

changelog: use a Factory for default value for files The default value is compiled into the generated type. This means that default values are shared between instances. For immutable types like bool, str, int, and tuple, this is fine. But for mutable types like list and dict, we need to use attr.Factory() to instantiate a new instance of the default for each object. Differential Revision: https://phab.mercurial-scm.org/D901

cext: wrap before brace for functions This is our prevailing style. Differential Revision: https://phab.mercurial-scm.org/D910

cext: put case statements on separate line This seems to be the prevailing style, even though it is a bit more verbose for very simple switch statements. Differential Revision: https://phab.mercurial-scm.org/D909

cext: reorder #include We mostly abide by this style. In one case, a blank line was inserted to prevent a local `#include "file"` from coming before a `#include <file>`. Differential Revision: https://phab.mercurial-scm.org/D908

cext: move braces for control statements to same line This seems to be the prevailing style in the code by a wide margin. Differential Revision: https://phab.mercurial-scm.org/D907

registrar: fixing typo in comment I was just going through the module and noticed the typo. This commit fixes 'onfalure' -> 'onfailure'. Differential Revision: https://phab.mercurial-scm.org/D906

style: never put multiple statements on one line Differential Revision: https://phab.mercurial-scm.org/D905

annotate: mark lines affected by skip-annotate with * This is to prevent weird surprises from happening with skips being attributed to the wrong changeset. .. feature:: `hg annotate --skip` now prints a `*` on lines with skipped revisions Differential Revision: https://phab.mercurial-scm.org/D900

annotate: track whether a particular annotation was the result of a skip We're going to expose this information in the UI in an upcoming patch. Differential Revision: https://phab.mercurial-scm.org/D899

annotate: introduce attr for storing per-line annotate data We're going to extend this a bit -- at first by simply adding whether this was a skipped child. We're well on our way to outgrowing tuples, though -- adding more and more fields to tuples becomes annoying very quickly. Differential Revision: https://phab.mercurial-scm.org/D898

context: rename local 'attr' to 'attr_' In the next diff we're going to import mercurial.thirdparty.attr, and pyflakes complains about this if this rename isn't done. Differential Revision: https://phab.mercurial-scm.org/D897

annotate: move annotatepair unit tests to a separate file In upcoming patches the output is going to be significantly longer than it is today, and doctests don't allow wrapping the output. Differential Revision: https://phab.mercurial-scm.org/D896

check-code: allow an exception for camelcase where required unittest has a `maxDiff` parameter which has to be set to `None` in order for large enough failure diffs to be displayed. Add a comment to disable the camelcase check for `self.maxDiff = None` lines. Differential Revision: https://phab.mercurial-scm.org/D895

url: use native strings for header values Differential Revision: https://phab.mercurial-scm.org/D889

keepalive: python 3 portability tweaks Differential Revision: https://phab.mercurial-scm.org/D888

httppasswordmgrdbproxy: specify exact arguments We only ever call these functions in a single way, so let's just actually specify them. We need to do some string/bytes encoding dancing here for Python 3, so it'll help to know what arguments we need to convert. # no-check-commit because I'm modifying functions that check-commit does not like. Differential Revision: https://phab.mercurial-scm.org/D885

formatter: fix default list/dict generator to be evaluated more than once Before, _hybrid.gen must be a generator which could be consumed only once. It was okay in templatekw.py since template keywords are functions which create temporary hybrid objects, but the formatter doesn't work in that way. To work around the issue, this patch makes _hybrid.gen optionally be a function returning a generator. Thanks to Pulkit for finding this issue.

doctest: drop hack to run py2/3 tests selectively All doctests pass on Python 3.

docker: try to follow the best practices for writing Dockerfiles Merged multiple RUN instructions and sorted the arguments alphabetically Reference: https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/

effectflag: document effect flag Differential Revision: https://phab.mercurial-scm.org/D542

effectflag: detect when diff changed Store in effect flag when the diff changed between the predecessor and its successors. Comparing the diff is not easy because we do not want to incorrectly detect a

effectflag: detect when meta changed Store in effect flag when the meta changed between the predecessor and its successors. We blacklisted some known meta that would always changed when another flag change. For example rebase would always add a meta rebase-source while the effect flag parents will already detect this situation. It can happens with various hg commands. Differential Revision: https://phab.mercurial-scm.org/D540

effectflag: detect when parents changed Store in effect flag when the parents changed between the predecessor and its successors. It can happens with "hg rebase" or "hg grab". Differential Revision: https://phab.mercurial-scm.org/D539

effectflag: detect when branch changed Store in effect flag when the branch changed between the predecessor and its successors. It can happens with "hg branch" + "hg commit --amend", "hg branch" + "hg amend" or "histedit". Differential Revision: https://phab.mercurial-scm.org/D538

effectflag: detect when date changed Store in effect flag when the date changed between the predecessor and its successors. It can happens with "hg commit --amend -d", "hg amend -d" or "histedit". Differential Revision: https://phab.mercurial-scm.org/D537

effectflag: detect when user changed Store in effect flag when the user changed between the predecessor and its successors. It can happens with "hg commit --amend -u" or "histedit". Differential Revision: https://phab.mercurial-scm.org/D536

effectflag: detect when description changed Store in effect flag when the description changed between the predecessor and its successors. It can happens with "hg commit --amend -e", "hg amend -e" or "histedit". Differential Revision: https://phab.mercurial-scm.org/D535

tests: add tests for effect flags Add all the tests in this patch, it makes the patch quite big but will clarify the following patches impact on tests. Differential Revision: https://phab.mercurial-scm.org/D534

effectflag: store an empty effect flag for the moment The idea behind effect flag is to store additional information in obs-markers about what changed between a changeset and its successor(s). It's a low-level information that comes without guarantees. This information can be computed a posteriori, but only if we have all changesets locally. This is not the case with distributed workflows where you work with several people or on several computers (eg: laptop + build server). Storing the effect-flag as a bitfield has several advantages: - It's compact, we are using one byte per obs-marker at most for the effect- flag. - It's compoundable, the obsfate log approach needs to display evolve history that could spans several obs-markers. Computing the effect-flag between a changeset and its grand-grand-grand-successor is simple thanks to the bitfield. The effect-flag design has also some limitations: - Evolving a changeset and reverting these changes just after would lead to two obs-markers with the same effect-flag without information that the first and third changesets are the same. The effect-flag current design is a trade-off between compactness and usefulness. Storing this information helps commands to display a more complete and understandable evolve history. For example, obslog (an Evolve command) use it to improve its output: x 62206adfd571 (34302) obscache: skip updating outdated obscache... | rewritten(parent) by Matthieu Laneuville <matthieu.laneuville@octobus... | rewritten(content) by Boris Feld <boris.feld@octobus.net> The effect flag is stored in obs-markers metadata while we iterate on the information we want to store. We plan to extend the existing obsmarkers bit-field when the effect flag design will be stabilized. It's different from the CommitCustody concept, effect-flag are not signed and can be forged. It's also different from the operation metadata as the command name (for example: amend) could alter a changeset in different ways (changing the content with hg amend, changing the description with hg amend -e, changing the user with hg amend -U). Also it's compatible with every custom command that writes obs-markers without needing to be updated. The effect-flag is placed behind an experimental flag set to off by default. Hook the saving of effect flag in create markers, but store only an empty one for the moment, I will refine the values in effect flag in following patches. For more information, see: https://www.mercurial-scm.org/wiki/ChangesetEvolutionDevel#Record_types_of_operation Differential Revision: https://phab.mercurial-scm.org/D533

configitems: register the 'profiling.type' config

configitems: register the 'profiling.showmin' config

configitems: register the 'profiling.showmax' config

configitems: register the 'profiling.output' config

dirstate: implement __len__ on dirstatemap (issue5695) Differential Revision: https://phab.mercurial-scm.org/D884

obsmarker: crash more helpfully when metadata fields are >255bytes (issue5681) Various mutators fail when attempting to write obsmarkers with metadata fields longer than 255 bytes, since the length of mwetadata fields is stored in u8s. This change raises a more helpful error in such circumstances. Differential Revision: https://phab.mercurial-scm.org/D865

deb: build and install chg Differential Revision: https://phab.mercurial-scm.org/D883

deb: install zsh completions to /usr/share/zsh/vendor-completions This location is used by debian (and ubuntu) to store completions provided by other deb packages. The default fpath appears to have this before any of the zsh-provided instances of the completions, so this should take precedence. Differential Revision: https://phab.mercurial-scm.org/D882

releasenotes: display release notes when no filename is specified If the filename is not specified while using --rev, the notes for the specified revs will just be displayed on screen. Differential Revision: https://phab.mercurial-scm.org/D877

hgweb: use parsebool for parsing diff query string options Differential Revision: https://phab.mercurial-scm.org/D875

hgweb: remove extra </div> This was accidentally added in 6797f1fbc642. Differential Revision: https://phab.mercurial-scm.org/D880

tests: add "have" check for dpkg builddeps Differential Revision: https://phab.mercurial-scm.org/D879

tests: expect parsers.so in cext, parsers.py in pure (test-debian-packages) Differential Revision: https://phab.mercurial-scm.org/D878

hghave: check for debuild being installed as well Differential Revision: https://phab.mercurial-scm.org/D874

changelog: use attrs instead of namedtuple See http://www.attrs.org/en/stable/why.html#namedtuples for why attrs are better than namedtuples. Differential Revision: https://phab.mercurial-scm.org/D868

thirdparty: vendor attrs The attrs package allows defining namedtuple-like classes with no weird behavior and no runtime performance cost. This patch vendors in attrs 17.2.0. # no-check-commit Differential Revision: https://phab.mercurial-scm.org/D867

python3: don't byte mangle third-party packages Third-party packages are already expected to be dual-version clean. Differential Revision: https://phab.mercurial-scm.org/D871

tests: disable lints on mercurial/thirdparty In the next patch, this directory will be used to vendor in some third-party code. Differential Revision: https://phab.mercurial-scm.org/D866

hghave: fix dpkg --version check to work on recent dpkg versions Differential Revision: https://phab.mercurial-scm.org/D869

commands: rename clone --uncompressed to --stream and document --uncompressed isn't a very good name and its description in the help documentation isn't very useful. We refer to this concept as "stream clones" in a number of places. I think it makes sense to change the user-facing argument to use the mode --stream. So this commit does that. We keep --uncompressed around for backwards compatibility. While we're here, we overhaul the help docs for streaming clones to be somewhat useful. All tests have been updated to reflect the new preferred --stream argument. A test for backwards compatibility of --uncompressed has been added. .. bc:: `hg clone --stream` should now be used instead of --uncompressed. --uncompressed is marked as deprecated and is an alias for --stream. There is no schedule for elimination of --uncompressed. Differential Revision: https://phab.mercurial-scm.org/D864

commands: remove suggestion to clone via `cp -al` This was added in 2df98f616645 in 2006. This may be possible but it seems like a bad idea to even suggest it. Differential Revision: https://phab.mercurial-scm.org/D863

hgweb: add HTML elements to control whitespace settings for annotate Building on top of the new URL query string arguments to control whitespace settings for annotate, this commit adds HTML checkboxes reflecting the values of these arguments to the paper and gitweb themes. The actual diff settings are now exported to the templating layer. The HTML templates add these as data-* attributes so they are accessible to the DOM. A new <form> with various <input> elements is added. The <form> is initially hidden via CSS. A shared JavaScript function (which runs after the <form> has been rendered but before the annotate HTML (because annotate HTML could take a while to load and we want the form to render quickly) takes care of setting the checked state of each box from the data-* attributes. It also registers an event handler to modify the URL and refresh the page whenever the checkbox state is changed. I'm using the URLSearchParams interface to perform URL manipulation. https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams tells me this may not be supported on older web browsers. Yes, apparently the web API didn't have a standard API to parse and format query strings until recently. Hence the check for the presence of this feature in the JavaScript. If the browser doesn't support the feature, the <form> will remain hidden and behavior will like it currently is. We could polyfill this feature or implement our own query string parsing. But I'm lazy and this could be done as a follow-up if people miss it. We could certainly expand this feature to support more diff options (such as lines of context). That's why the potentially reusable code is stored in a reusable place. It is also certainly possible to add diff controls to other pages that display diffs. But since Mozillians are making noise about controlling which revisions annotate shows, I figured I'd start there. .. feature:: Control whitespace settings for annotation on hgweb /annotate URLs on hgweb now accept query string arguments to influence how whitespace changes impact results. The arguments "ignorews," "ignorewsamount," "ignorewseol," and "ignoreblanklines" now have the same meaning as their [annotate] config section counterparts. Any provided setting overrides the server default. HTML checkboxes have been added to the paper and gitweb themes to expose current whitespace settings and to easily modify the current view. Differential Revision: https://phab.mercurial-scm.org/D850

hgweb: query string arguments to control whitespace for annotate This feature should hopefully be pretty straightforward. We simply examine some query string arguments to feed into the diff options. The function to obtain the diff options has been factored into its own generic function to facilitate an upcoming change to the HTML interface and to enable diff settings to be controlled via the same query string arguments on other web commands. The test output for "ignoreblanklines" didn't change. I'm not sure why. Our test coverage for --ignore-blank-lines isn't great and I can't figure out how to make this diff setting do anything meaningful. On a very brief examination of the code, it is possible the setting doesn't work because it is operating at the line level and blank lines detection needs to examine multiple lines. But I'm not an expert in this code, so I'm not sure. Differential Revision: https://phab.mercurial-scm.org/D849

tests: add tests for hgweb annotate output In preparation for adding features to modify whitespace settings. Differential Revision: https://phab.mercurial-scm.org/D848

obsolete: clean createmarkers part about operation I will add another experiment in createmarkers, add a comment and some blank lines for aesthetic sake. Differential Revision: https://phab.mercurial-scm.org/D532

treediscovery: update test expectation I'm not sure why this needs an update, but the test passes for me both pure and non-pure. Differential Revision: https://phab.mercurial-scm.org/D860

tests: update some clonebundles expectations to pass on both pure and non-pure Specifically, zstd isn't available in --pure mode, and we need to glob off the leading comma as well as the absent compression engine. Differential Revision: https://phab.mercurial-scm.org/D859

merge with stable

contrib: add a check to check-code to ban superfluous pass statements These have annoyed me for a long time, and I'm tired of commenting on them in reviews. I'm sorry for how complicated the regular expression is, but I was too lazy to go crack open pylint's code and add the check there.

posix: replace superfluous pass statement with explicit return

subrepo: remove superfluous pass statements

registrar: remove superfluous pass statements

patch: remove superfluous pass statements

obsolete: remove superfluous pass statements

match: remove superfluous pass statements

localrepo: remove superfluous pass statements

hgweb: remove superfluous pass statements

formatter: remove superfluous pass statements

exchange: remove superfluous pass statements

commandserver: remove superfluous pass statements

cmdutil: remove superfluous pass statements

byterange: remove superfluous pass statements

branchmap: remove superfluous pass statements

convert: remove superfluous pass statements

bugzilla: remove superfluous pass statements

merge with stable

copytrace: use ctx.mutable() instead of adhoc constant of non-public phases

exchange: fix test for remote support of binary phases If the remote do not support phases, the "get" call will return None. We change that default return to empty tuple to fix the membership testing. This was a bug in the initial series. Thanks to yuja for catching this.

exchange: perform stream clone with clone bundle with --uncompressed Previously, `hg clone --uncompressed` would always clone from the origin server, even if a streaming clone bundle were available. With this change, we invoke the clone bundle mechanism before the stream clone mechanism, giving clone bundles the opportunity to handle --uncompressed (which is mapped to pullop.streamclonepreferred). The clone bundle filtering code now filters out entries that aren't stream clones when a stream clone is requested. If a stream clone clone bundle entry is present, it will be used. Otherwise, the client will fall back to a server-based streaming clone. .. feature:: `hg clone --uncompressed` uses clone bundles when possible Differential Revision: https://phab.mercurial-scm.org/D833

tests: add tests for clone bundles with --uncompressed Currently, --uncompressed will always clone from the origin server, even if a stream clone bundle is present. Let's add tests demonstrating this behavior. Differential Revision: https://phab.mercurial-scm.org/D832

py3: work around bytes/unicode divergence in parsedate()

py3: replace bytes[n] with slicing in checkwinfilename()

py3: manually escape control character to be embedded in win filename error

py3: replace str(None) with literal in convcmd.py

py3: remove use of str() in hgwebdir '%d' can't be used here since port may be either integer or byte string.

config: use copy-on-write to improve copy performance Previously, chg's `verify` call could take 30+ms loading and checking new config files. With one socket redirection, that adds up to around 70ms, which is a lot for fast commands (ex. `bookmark --hidden`). When investigating closer, A lot of time was spent on actually spent on ui copying, which is mainly about `config.config` and `dict` copying. This patch makes that 20x faster by adopting copy-on-write. The copy-on-write is performed at config section level. Before: In [1]: %timeit ui.copy() 100 loops, best of 3: 2.32 ms per loop After: In [1]: %timeit ui.copy() 10000 loops, best of 3: 128 us per loop 2ms may look not that bad, but it adds up pretty quickly with multiple calls. A typical chg run may call it 4 times, which is about 10ms. Differential Revision: https://phab.mercurial-scm.org/D808

py3: whitelist 15 more tests passing on Python 3 We have reached the count of 75 tests passing on Python 3. Differential Revision: https://phab.mercurial-scm.org/D858

py3: use pycompat.bytestr instead of str Differential Revision: https://phab.mercurial-scm.org/D855

py3: explicitly convert dict.keys() and dict.items() into a list Differential Revision: https://phab.mercurial-scm.org/D853

py3: use '%d' instead of '%s' for integers Differential Revision: https://phab.mercurial-scm.org/D856

py3: use pycompat.strkwargs() before passing a dict as keyword argument Differential Revision: https://phab.mercurial-scm.org/D854

py3: return False early while checking whether None is a key in lazymanifest In this patch we returns False early if we are trying to check whether None is a key in lazymanifest. The reason I added a diff is that on Python 3, it goes into the lazy manifest code an returns a TypeError. I checked with Durham that whether None can be a possible key in lazymanifest and he said "no". So it's safe to have this if statement. This fixes `hg merge` on Python 3. Differential Revision: https://phab.mercurial-scm.org/D852

dirstate: use keyword arguments to clarify status()'s callers The arguments are especially non-obvious because the order is different from dirstate.walk(). Differential Revision: https://phab.mercurial-scm.org/D847

dirstate: use keyword arguments to clarify walk()'s callers The arguments are especially non-obvious because the order is different from dirstate.status(). Differential Revision: https://phab.mercurial-scm.org/D846

perf: remove fallbacks to ancient versions of dirstate.walk() If the call to dirstate.walk() failed, we would try to fall back to older versions. These were removed in d3d1d39da2fa (walk: remove cmdutil.walk, 2008-05-12) and f8299c84b5b6 (dirstate: fold statwalk and walk, 2008-06-26). We don't care about testing performance of versions that old versions at this point, so let's clean up. Differential Revision: https://phab.mercurial-scm.org/D845

releasenotes: update docstrings with information on additional flags This patch adds documentation related to the two new flags --check and --list for releasenotes extension. Differential Revision: https://phab.mercurial-scm.org/D544

releasenotes: raise error on simultaneous usage of flags The releasenotes command is supposed to raise an error when --list and --rev/--check flags are used together. This patch adds the above functionality. Differential Revision: https://phab.mercurial-scm.org/D831

dirstate: move parents source of truth to dirstatemap As part of moving dirstate storage to its own class, let's move the source of truth for the dirstate parents to dirstatemap. This requires that dirstate._pl no longer be a cache, and that all sets go through dirstatemap.setparents. Differential Revision: https://phab.mercurial-scm.org/D759

dirstate: move parent reading to the dirstatemap class As part of moving dirstate storage logic to a separate class, let's move the function that reads the parents from the file. This will allow extensions to write dirstate's that store the parents in other ways. Differential Revision: https://phab.mercurial-scm.org/D758

dirstate: move opendirstatefile to dirstatemap As part of moving the dirstate storage logic to another class, let's move opendirstatefile to dirstatemap. This will allow extensions to replace the pending abstraction. Future patches will move the consumers of _opendirstatefile into dirstatemap as well. Differential Revision: https://phab.mercurial-scm.org/D757

dirstate: move _copymap to dirstatemap As part of moving all dirstate storage to a new class, let's move the copymap onto that class. In a future patch this will let us move the read/write functions to the dirstatemap class, and for extensions this lets us replace the copy storage with alternative storage. Differential Revision: https://phab.mercurial-scm.org/D756

dirstate: move _dirs to dirstatemap As part of moving the dirstate storage logic to a new class, lets move the _dirs computation onto the class so extensions can replace it with a persisted index of directories. Differential Revision: https://phab.mercurial-scm.org/D755

dirstate: move filefoldmap to dirstatemap As part of moving the dirstate storage logic to a separate class, lets move the filfoldmap computation to that class. This will allow extensions to replace the dirstate storage with something that persists the filefoldmap. Differential Revision: https://phab.mercurial-scm.org/D754

dirstate: move nonnormalentries to dirstatemap As part of moving dirstate storage to its own class, let's move the nonnormalentries logic onto the dirstatemap class. This will let extensions replace the nonnormalentries logic with a persisted cache. Differential Revision: https://phab.mercurial-scm.org/D753

dirstate: create new dirstatemap class This is part of a larger refactor to move the dirstate storage logic to a separate class, so it's easier to rewrite the dirstate storage layer without having to rewrite all the algorithms as well. Step one it to create the class, and replace dirstate._map with it. The abstraction bleeds through in a few places where the main dirstate class has to access self._map._map, but those will be cleaned up in future patches. Differential Revision: https://phab.mercurial-scm.org/D752

style: always use `x is not None` instead of `not x is None` Differential Revision: https://phab.mercurial-scm.org/D842

templatekw: add new-style template expansion to {manifest} The goal is to allow us to easily access to nested data. The dot operator will be introduced later so we can write '{p1.files}' instead of '{revset("p1()") % "{files}"}' for example. In the example above, 'p1' needs to carry a mapping dict along with its string representation. If it were a list or a dict, it could be wrapped semi-transparently with the _hybrid class, but for non-list/dict types, it would be difficult to proxy all necessary functions to underlying value type because several core operations may conflict with the ones of the underlying value: - hash(value) should be different from hash(wrapped(value)), which means dict[wrapped(value)] would be invalid - 'value == wrapped(value)' would be false, breaks 'ifcontains' - len(wrapped(value)) may be either len(value) or len(iter(wrapped(value))) So the wrapper has no proxy functions and its scope designed to be minimal. It's unwrapped at eval*() functions so we don't have to care for a wrapped object unless it's really needed: # most template functions just call evalfuncarg() unwrapped_value = evalfuncarg(context, mapping, args[n]) # if wrapped value is needed, use evalrawexp() maybe_wrapped_value = evalrawexp(context, mapping, args[n]) Another idea was to wrap every template variable with a tagging class, but which seemed uneasy without a static type checker. This patch updates {manifest} to a mappable as an example.

templater: adjust binding strength of '%' and '|' operators (BC) This makes 'foo|bar%baz' parsed as '(foo|bar)%baz', not 'foo|(bar%baz)'. Perhaps it was a mistake that '%' preceded '|'. Both '|' and '%' can be considered a kind of function application, and '|' is more like a '.' operator seen in OO languages. So IMHO '|' should have the same (or higher) binding as '%'. The BC breakage should be minimal since both '|' and '%' operators have strict requirements for their operands and 'foo|bar%baz' was invalid: - right-hand side of '|' must be a symbol - left-hand side of '%' must be a dict or list - right-hand side of '%' must be a string or symbol

templatekw: just pass underlying value (or key) to joinfmt() function Before, iter(hybrid) was proxied to hybrid.gen, which generated formatted strings. That's why we had to apply joinfmt() to the dicts generated by hybrid.itermaps(). Since this weird API was fixed at a0f2d83f8083, we can get rid of the makemap() calls from join().

scmutil: extract helper functions that returns human-readable change id We do "'%d:%s' % (ctx...)" at several places, so let's formalize it. A low- level function, formatrevnode(ui, rev, node), is extracted so we can pass a manifest rev/node pair. Note that hex() for manifest output can be replaced with hexfunc() because it is printed only when debugflag is set. i18n/de.po is updated so test-log.t passes with no error.

templater: extract helper to just evaluate template expression A named function can be easily grepped and is probably good for code readability.

templater: do not destructure operands in buildmap() This makes the next patch slightly simpler.

templater: use helper function to get name of non-iterable keyword

pull: remove inadequate use of operations records to update stepdone The 'stepdone' set is design to be a client side mechanism. If the client used some advanced capabilities to request necessary information (changeset, obsmarkers, phases, etc). It marks the steps as done to avoid having a less advanced mechanism issue a duplicated request. So, the "stepdone.add('phases')" should be the result of a client choice, because only the client can know it has requested all it needed to request. In 4a08cf1a2cfe this principle was broken because any phase-heads part sent by the server to the client would declare the phases retrieval complete. Now that there is an official phases related capability and code associated to it. We do not need the change in 4a08cf1a2cfe anymore and we can back it out. This brings back 'stepdone' management for 'phases' in line with the rest of the code (including other phases handing). Here is an example of potential misbehavior that 4a08cf1a2cfe introduced: Imagine a server that pre-computes bundles. The bundles contains a changegroup part and an (advisory) 'phase-heads' part. When a pull occurs, precomputed bundled are reused if available. As the phase part is advisory it can be sent to all clients. However they could be relevant changesets without phase information. Either because they are already common or because they had no precomputed bundle for them yet. If receiving any 'phase-heads' parts disable subsequent phases re-trivial parts, the client will not request phase data for all relevant changesets. For example common changesets will not turn public.

pull: use 'phase-heads' to retrieve phase information A new bundle2 capability 'phases' has been added. If 'heads' is part of the supported value for 'phases', the server supports reading and sending 'phase- heads' bundle2 part. Server is now able to process a 'phases' boolean parameter to 'getbundle'. If 'True', a 'phase-heads' bundle2 part will be included in the bundle with phase information relevant to the whole pulled set. If this method is available the phases listkey namespace will no longer be listed. Beside the more efficient encoding of the data, this new method will greatly improve the phase exchange efficiency for repositories with non-served changesets (obsolete, secret) since we'll no longer send data about the filtered heads. Add a new 'devel.legacy.exchange' config item to allow fallback to the old 'listkey in bundle2' method. Reminder: the pulled set is not just the changesets bundled by the pull. It also contains changeset selected by the "pull specification" on the client side (eg: everything for bare pull). One of the reason why the 'pulled set' is important is to make sure we can move -common- nodes to public.

bundle2: only grab a transaction when 'phase-heads' affect the repository The next patch will use the 'phase-heads' part to exchange phase data relevant to the pulled set. 'handlephases' currently acquires a transaction even in case of no-op pull, which would results in an empty transaction and messing with the existing journal. Pass the transaction fetcher to updatephases so it can fetch it if necessary.

phases: move the binary decoding function in the phases module We move the decoding function near the encoding one in a place where they can be reused in other place (current target, 'exchange.py').

phases: move binary encoding into a reusable function We want to use binary phases for pushing and pulling. We extract the encoding function out of the bundle2 module first.

phases: use a Struct object for binary encoding and decoding We will move the binary encoding and decoding code to 'phases.py' in order to make it easier to reuse. First, let's cleanup it a bit.

discovery: avoid dropping remote heads hidden locally An extra post processing was added to recognize remote heads that are hidden locally as "common" instead of "unknown". However, this processing was removing such hidden heads from the remote heads sets. It had no impact because we used to pull phase information from all remote heads. This series will replace the phase pulling operation to a more efficient process but requires the unmodified pulled set information.

test-patchbomb: use mocktime The test was using system time for displaying ETAs, which could be flaky if the sysload is high. This patch extracts mocktime.py from test-progress.t to make sure test-patchbomb.t is unaffected by system time. Differential Revision: https://phab.mercurial-scm.org/D844

test-patchbomb: fix the test With the experimental config `progress.estimate` removed, the progress output in `test-patchbomb.t` has a minor change: it shows ETA since the beginning. (This could be folded into f428c3)

progress: make ETA only consider progress made in the last minute This patch limits the estimate time interval to roughly the last minute (configurable by `estimateinterval`) to be more practical. See the test change for why this is better. .. feature:: Estimated time is more accurate with non-linear progress Differential Revision: https://phab.mercurial-scm.org/D820

progress: remove progress.estimate config It was introduced by 98e4d39 ("progress: add speed format" 2011-5-9) and was intended to hide ETA information for the first few seconds. Later 5d261fd ("progress: add a changedelay to prevent parallel topics from flapping (issue2698)" 2011-6-23) introduced `changedelay` config which hides the entire progress bar for the first few seconds. So `progress.estimate` seems somehow duplicated feature-wise. Since it's experimental and duplicated, let's just remove it. This makes the next patch simpler - it no longer needs to make sure `starttimes` is the real start time. Differential Revision: https://phab.mercurial-scm.org/D828

progress: demonstrate non-linear progress has a bad ETA experience Previously, the ETA and speed assumes the progress is linear. Often, due to network or other issues, it could be fast for the most time, and suddenly slow down: [====================================================> ] \___________________________________________/\______/ very fast suddenly much slower This patch adds a test demonstrating the ETA could be way off in those cases. Differential Revision: https://phab.mercurial-scm.org/D819

copytrace: add a a new config to limit the number of drafts in heuristics The heuristics options tries to the default full copytracing algorithm if both the source and destination branches contains of non-public changesets only. But this can be slow in cases when we have a lot of drafts. This patch adds a new config option experimental.copytrace.sourcecommitlimit which defaults to 100. This value will be the limit of number of drafts from c1 to base. Incase there are more changesets even though they are draft, the heuristics algorithm will be used. Differential Revision: https://phab.mercurial-scm.org/D763

mail: encode long unicode lines in emails properly (issue5687) 3e544c074459 introduced a bug: emails Content-Transfer-Encoding is silently replaced with 'quoted-printable' while any other encoding could be used by underlying code. The problem is revealed when a long unicode line is encoded. The patch implements proper check which works for any text and encoding.

chg: show timestamp with debug messages Like `strace -tr`, this helps finding performance bottlenecks. Differential Revision: https://phab.mercurial-scm.org/D807

keepalive: add more context to bad status line errors As the TODO in the test said, the previous error message was not very helpful. Let's improve things. Differential Revision: https://phab.mercurial-scm.org/D811

tests: add interface checks for bundle, statichttp, and union peers I forgot to add these when I initially wrote the test. They inherit from localrepo.localpeer, so they should be explicitly tested. Differential Revision: https://phab.mercurial-scm.org/D810

alias: make alias command lazily resolved With many aliases, resolving them could have some visible overhead. Below is part of traceprof [1] output of `hg bookmark --hidden`: (time unit: ms) 37 \ addaliases dispatch.py:526 37 | __init__ (60 times) dispatch.py:402 33 | findcmd (108 times) cmdutil.py:721 16 | findpossible (49 times) cmdutil.py:683 It may get better by optimizing `findcmd` to do a bisect, but we don't really need to resolve an alias if it's not used, so let's make those command entries lazy. After this patch, `addalias` takes less than 1ms. .. perf:: improved performance when many aliases are defined [1]: https://bitbucket.org/facebook/hg-experimental/src/9aca0dbdbdfc48457e5d2581ca2d6e662fced2e6/hgext3rd/traceprof.pyx Differential Revision: https://phab.mercurial-scm.org/D805

alias: test duplicated definition earlier This patch moves the old definition checking logic introduced by f4b7be3f8430 earlier. So that the test itself does not depend on `aliasdef`. The check is to avoid wrapping a same alias multiple times. It can be done by checking the config name and value (`definition` in code), without constructing a `cmdalias` instance. This makes the next patch easier to review. Differential Revision: https://phab.mercurial-scm.org/D804

uncommit: add a test for uncommit with uncommitondirtywdir config in merge Differential Revision: https://phab.mercurial-scm.org/D809

largefiles: force an on-disk merge Largefiles isn't a good candidate for in-memory merge (it uses a custom dirstate, matcher, and the files might not fit in memory) so have it always run an old-style merge. Differential Revision: https://phab.mercurial-scm.org/D683

merge: allow a custom working context to be passed to update This will allow anyone to enable the first in-menmory merge milestone by wrapping merge.update in an extension and creating an overlayworkingctx. Differential Revision: https://phab.mercurial-scm.org/D682

patchbomb: add test that shows --to and --cc override matching config item As far as I know this has always been true and is intentional (it's in line with many other behaviors), but it wasn't tested.

blackbox: move _openlogfile to a separate method This removes several `stat` syscalls for accessing `_bbvfs` and makes `ui` object cleaner. Differential Revision: https://phab.mercurial-scm.org/D769

blackbox: do not prevent 'chg init' from working Previously, blackbox always appends to blackbox.log and creates the directory for that file on demand. That could be an issue if: 1. chg starts from `$REPO` directory, so `ui._bbrepo` is set. 2. `rm -rf $REPO`. 3. `chg init $REPO`, blackbox writes something and `init` will fail because `$REPO` directory is non-empty. This patch fixes that by verifying whether vfs exists before re-using it. Differential Revision: https://phab.mercurial-scm.org/D768

tests: add a test for blackbox with nested alias configurations I've observed some weirdness around this, and needed to rule some things out. There aren't any bugs in core around this, but it was nice to have confirmation.

tests: clean up blackbox test around aliases a little bit

revlog: add option to mmap revlog index Following on from Jun Wu's patch last October[1], we have found that using mmap for the revlog index in repos with large revlogs gives a noticable performance improvment (~110ms on each hg invocation), particularly for commands that don't touch the index very much. This changeset adds this as an option, activated by a new experimental config option so that it can be enabled on a per-repo basis. The configuration option specifies an index size threshold at which Mercurial will switch to using mmap to access the index. If the configuration option is not specified, the default remains to load the full file, which seems to be the best option for smaller repos. Some initial performance numbers for average of 5 invocations of `hg log -l 5` for different cache states: | Repo: | HG | FB | |---|---|---| | Index size: | 2.3MB | much bigger | | read (warm): | 237ms | 432ms | | mmap (warm): | 227ms | 321ms | | | (-3%) | (-26%) | | read (cold): | 397ms | 696ms | | mmap (cold): | 410ms | 888ms | | | (+3%) | (+28%) | [1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-October/088737.html Test Plan: `hg log --config experimental.mmapindex=true` Differential Revision: https://phab.mercurial-scm.org/D477

util: add an mmapread method This is useful for large files that are only partly touched. Test Plan: Will be used and tested in a later patch. Differential Revision: https://phab.mercurial-scm.org/D476

changegroup: remove dictionary creation from deltachunk Previously delta chunk returned a dictionary. Now that we consume deltachunk within changegroup (instead of outside in revlog) we can just return a tuple and have it be returned directly by deltaiter. Differential Revision: https://phab.mercurial-scm.org/D746

bundlerepo: update to use new deltaiter api Differential Revision: https://phab.mercurial-scm.org/D745

debug: update debugbundle to use new deltaiter api Changegroup now has a deltaiter api for easy iteration over a series of deltas. Let's use that in the debugbundle command. Differential Revision: https://phab.mercurial-scm.org/D744

revlog: add revmap back to revlog.addgroup The recent c8b6ed51386b patch removed the linkmapper argument from addgroup, as part of trying to make addgroup more agnostic from the changegroup format. It turns out that the changegroup can't resolve linkrevs while iterating over the deltas, because applying the deltas might affect the linkrev resolution. For example, when applying a series of changelog entries, the linkmapper just returns len(cl). If we're iterating over the deltas without applying them to the changelog, this results in incorrect linkrevs. This was caught by the hgsql extension, which reads the revisions before applying them. The fix is to return linknodes as part of the delta iterator, and let the consumer choose what to do. Differential Revision: https://phab.mercurial-scm.org/D730

rebase: also include other namespaces in changeset description This makes use of the generic method of listing bookmarks and tags, so other extensions that add other namespaces will get their names added too. This does mean that bookmarks will come before tags, just like we apparently decided to order them in the "hg log" output. It doesn't seem like people would be parsing the rebase output anyway. We also did 79ab5369d55a (rebase: use _ctxdesc in one more place, 2017-08-29) recently, so now seems like a good time. Differential Revision: https://phab.mercurial-scm.org/D741

tersestatus: sort the dictionary before iterating on it There has report of flakiness in test-status-terse.t. In the terse code, we are iterating on a dictionary without sorting and since python dicts are unordered, that can be a reason behind the flakiness. Before we have a better implementation for the terse thing, let's make sure current implementation possess minimum bugs. Differential Revision: https://phab.mercurial-scm.org/D740

copytrace: use the full copytracing method if only drafts are involved This patch adds the functionality to use the full copytracing even if `experimental.copytrace = heuristics` in cases when drafts are involved. This is also a part of copytrace extension in fbext. This also adds tests which are also taken from fbext. .. feature:: The `heuristics` option for `experimental.copytrace` performs full copytracing if both source and destination branches contains non-public changsets only. Differential Revision: https://phab.mercurial-scm.org/D625

templates: introduce a obsfateoperation() function Differential Revision: https://phab.mercurial-scm.org/D723

obsmarker: track operation by default We added support for including the operation responsible for creating the obsmarker in 3546a771e376 (obsolete: add operation metadata to rebase/amend/histedit obsmarkers, 2017-05-09). However, soon thereafter, in f432897a9f49 (obsmarker: add an experimental flag controlling "operation" recording, 2017-05-20), it was hidden behind a config that was off by default. It seems unlikely that people will manually turn it on, and obsmarkers/evolution as a whole is still experimental anyway, so let's turn on the tracking by default. Differential Revision: https://phab.mercurial-scm.org/D722

uncommit: add an experimental.uncommitondirtywdir config There are people who wants to do `hg uncommit` on dirty working directory, so this patch adds a config which can be used to the achieve that. Adds tests for the same. Differential Revision: https://phab.mercurial-scm.org/D735

uncommit: don't allow bare uncommit on dirty working directory As per IRC discussion and suggestion by Augie, we should not allow bare `hg uncommit` on dirty working directory. Differential Revision: https://phab.mercurial-scm.org/D734

uncommit: rename the flag 'empty' to 'keep' which retains empty changeset This patch renames the flag 'empty' which retains the empty changeset if all the files are uncommitted to 'keep'. Differential Revision: https://phab.mercurial-scm.org/D726

tests: avoid repr in test-bundle2-format.t We don't *really* need it here, and it was making Python 3 harder.

tests: port test-bundle2-format inline helper script to Python 3 We're now very close to this test passing.

bundle2: portably grab first byte of part name for letter check

bundle2: make ValueError messages native strings

bundle2: update check for a generator to work on Python 3

bundle2: stop using %r to quote part names Valid part names are restricted to [a-zA-Z0-9_:-]+, so I'm not worried about having quoting present in places where we should have predominantly valid part names. This will significantly ease the Python 3 transition, and simultaneously isn't a BC because this is only in error messages that should never be shown.

blackbox: set lastui even if ui.log is not called (issue5518) `lastui` decides where (where is the `.hg`) to use if the current `ui` object does not have a `_bbrepo` associated. Previously it only gets set in `ui.log`, which means unless a `ui` with repo associated calls `log` with tracked event, blackbox does not know where to write its log. This patch makes `reposetup` set `lastui` so it so we could log some more events (see test changes). Differential Revision: https://phab.mercurial-scm.org/D655

blackbox: unindent a try block The try block is no longer necessary. Differential Revision: https://phab.mercurial-scm.org/D654

blackbox: simplify ui states It seems cleaner to just remove `_partialinit`, `copy`, `__init__`. This patch makes it so by using `getattr` in `log` so those fields do not need to be existed. Differential Revision: https://phab.mercurial-scm.org/D652

revset: move weight information to predicate Previously revset weight is hardcoded and cannot be modified. This patch moves it to predicate so newly registered revsets could define their weight to properly give static optimization some hint. Differential Revision: https://phab.mercurial-scm.org/D657

revset: remove "small" argument from "_optimize" `_optimize` calculates weights of subtrees. "small" affects some weight calculation (either 1 or 0.5). The weights are now only useful in `and` optimization where we might swap two arguments and use `andsmally`. In the real world, it seems unlikely that revsets with weight of 0.5 or 1 matters the `and` order optimization. I think the important thing is to get weights of expensive revsets right (ex. `contains`). This patch removes the `small` argument to simplify the interface. As for choosing between 0.5 vs 1, things returning a single revision (`ancestor`, `string`) has a weight of 0.5. Things returning multiple revisions returns 1. This could be sometimes useful in the `andsmally` optimization, ex. (((:)-2) & expensive()) & ((1-2) & expensive()) ^^^ ^ ^^^^^^^ ^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^ weight=1 weight=0.5 would have an `andsmally` optimization so `1-2` gets executed first, which seems to be desirable. Differential Revision: https://phab.mercurial-scm.org/D656

ui: fix progress debug log format strings to work on Python 3

tests: update `f` helper script to work on Python 3

tests: fix run-tests XML reporting on Python 3 cdatasafe wants to work in terms of bytes, but of course we have a unicode. Easy to work around, especially since we know we'll get utf-8 at the end.

tests: convert bisect support regex to binary The bisection feature of run-tests still fails tests with this because bisect itself doesn't work yet. We'll get there.

tests: decode bytes path to hg command before hitting with shellquote

tests: fix test-is-whitelisted check in run-tests Again, look for bytes in a set of bytes objects.

tests: fix run-tests "slow test" check Look for bytes in a set of bytes objects.

tests: fix run-tests default values in Test constructor As far as I can tell, default values are evaluated far earlier on Python 3.6 than 2.7, meaning we've got to be more careful about when we read the defaults dictionary for the kwarg defaults. Sigh. With this change, test-run-tests.t is significantly less of a mess under 3.6.

tests: add support for listing tests to run in a file This will make it easier for us to run the tests in the python3 whitelist in buildbot.

python3: remove test-update-dest.t from the whitelist I've poked at this on and off several times, and I can't figure out what regressed. Let's kick this out of the whitelist for now so that we can get the *rest* of our progress covered by the buildbots.

bundle2: move part processing to a separate function Now that the part processing loop is tiny, let's move it to a separate function. This will allow extensions to completely replace the part processing logic, without having to replace the overall bundle processing logic or the stream maintenance logic. This will be useful for the infinitepush extension, so it can completely take over receiving a bundle and rerouting it to a side store. This will also make it easier to upstream the infinitepush functionality later. Differential Revision: https://phab.mercurial-scm.org/D709

bundle2: remove unnecessary try finally This is no longer needed. Differential Revision: https://phab.mercurial-scm.org/D708

bundle2: move handler validation out of processpart As part of refactoring bundle part processing let's move handler validation to its own function. Differential Revision: https://phab.mercurial-scm.org/D707

bundle2: move processpart stream maintenance into part iterator The processpart function also did some stream maintenance, so let's move it to the part iterator as well, as part of moving all part iteration logic into the class. There is one place processpart is called outside of the normal loop, so we manually handle the seek there. The now-empty try/finally will be removed in a later patch, for ease of review. Differential Revision: https://phab.mercurial-scm.org/D706

merge with stable

py3: convert system strings to bytes in doctest of formatter.py

py3: rewrite stdout hack of doctest by using ui.pushbuffer() We can't use pycompat.stdout here because the doctest runner replaces sys.stdout with a string buffer.

py3: use bytes os.sep in doctest of pathutil.py

py3: fix doctests in patch.py to be compatible with Python 3 We were lucky that parsepatch() could concatenate a character slice as if it were a list of chunks.

py3: stop using bytes[n] in patch.py

error: move patch.PatchError so it can easily implement __bytes__ (API)

templater: extract shortest() logic from template function It can be useful for extensions to be able to produce the shortest unambiguous hash (including the in-tree "show" extension). That logic is currently inside the shortest() template function. Let's move it out of the templater. I've put it on revlog since it's closely related to revlog._partialmatch. We may also want a convenience method on context, but I'll leave that for a later patch. Differential Revision: https://phab.mercurial-scm.org/D724

bundle2: raise a more helpful error if building a bundle part header fails I've tripped on this several times now, and am tired of debugging. Now the header parts are part of the error message when the ''.join() fails, which makes debugging obvious.

hgwebdir: read 'web.template' untrusted The 'hgweb_mod.py' version of this read it untrusted. For consistency we align the two versions of this code.

configitems: register the 'web.templates' config

configitems: register the 'web.style' config

configitems: register the 'web.stripes' config

configitems: register the 'web.refreshinterval' config