revlog: pass revlog flags to addrevision Adding the ability to passing flags to addrevision instead of simply passing default flags to _addrevision will allow extensions relying on flag transforms to wrap around addrevision() in order to update revlog flags. The first use case of this patch will be the lfs extension marking nodes as stored externally when the contents are larger than the defined threshold. One of the reasons leading to setting flags in addrevision() wrappers in the flag processor design is that it allows to detect files larger than the 2GB limit before the check is performed, which allows lfs to transform the contents into metadata.

revlog: add 'raw' argument to revision and _addrevision This patch introduces a new 'raw' argument (defaults to False) to revlog's revision() and _addrevision() methods. When the 'raw' argument is set to True, it indicates the revision data should be handled as raw data by the flagprocessor. Note: Given revlog.addgroup() calls are restricted to changegroup generation, we can always set raw to True when calling revlog._addrevision() from revlog.addgroup().

pager: do not special case chg Since chg has its own _runpager implementation, it's no longer necessary to special-case chg in the pager extension. This will effectively enable the new chg pager code path that runs inside runcommand.

chg: remove getpager support We have enough bits to switch to the new chg pager code path in runcommand. So just remove the legacy getpager support. This is a red-only patch, and will break chg's pager support temporarily.

chgserver: implement chgui._runpager This patch implements chgui._runpager in a relatively simple way. A more clean way is to move the core logic of "attachio" to "ui", which will be done later after chg runs uisetup per request.

chgserver: make S channel support pager request This patch adds the "pager" support for the S channel. The pager API allows running some subcommands, namely attachio, and waiting for the client to be properly synchronized.

chg: handle pager request client-side This patch implements the simple S-channel pager handling at chg client-side. Note: It does not deal with environ and cwd currently for simplicity, which will be fixed later.

chgserver: use util.shellenviron This avoids code duplication.

util: extract the logic calculating environment variables The method will be reused in chgserver. Move it out so it can be reused.

hgweb: generate archive links in order It would be nice for archive links to always be in a certain commonly used order, such as 'zip', 'bz', 'gzip2'. Repo index page (hgwebdir_mod) already shows archive links in this order, let's do the same in hgweb_mod. Sadly, archivespecs is a regular unordered dict, and collections.OrderedDict is new in 2.7. But requestcontext.archives is a tuple of archive types, so it can be used as an index to archivespecs.

hgweb: use archivespecs (dict) instead of archives (tuple) for "in" check

test-obsolete: stabilize output on platforms without 'serve' support The conditional was updating the repository, which wasn't reflected in subsequent logs on Windows, so the conditional is narrowed to just the serve commands. The serve operation generates log files, so those are deleted to keep the output of summary consistent.

tests: update globs for Windows The extra glob in test-command-template.t caused it to say no result was reported. It used to be (within the past year), that both this and the missing glob cases could be fixed simply by editing any output in the test, and re-running it in interactive mode. But that no longer works, and I had to diff *.t against *.t.err. I didn't dig into what changed.

help: merge the various operator sections of revsets, filesets and templates Having sections for specific operator types assumes the user already knows what type of operators are supported. By having a common heading, the user can simply lookup help for "(revsets|filesets|templates).operators".

help: apply the section headings from revsets to templates Unlike filesets, there are a few distinct headings that are not shared with revsets. But common names are used where possible.

help: apply the section headings from revsets to filesets This has the nice property of visually breaking up the wall of text. It also allows specific smaller sections to be called out. For example, `hg help filesets.predicates` now prints just the predicate section. At the moment, the revset headings are a superset of the fileset headings, so there is consistency in how example, predicate and operator help is called out. The reference to `hg help patterns` was moved to the overview section, so that it isn't stuck in the examples section.

chg: check type read from S channel The previous patch added the check server-side. This patch added it client-side.

chgserver: check type passed to S channel It currently only supports the "system" type. Add an explicit check.

chg: send type information via S channel (BC) Previously S channel is only used to send system commands. It will also be used to send pager commands. So add a type parameter. This breaks older chg clients. But chg and hg should always come from a single commit and be packed into a single package. Supporting running inconsistent versions of chg and hg seems to be unnecessarily complicated with little benefit. So just make the change and assume people won't use inconsistent chg with hg.

rebase: fail-fast the pull if working dir is not clean (BC) Refuse to run 'hg pull --rebase' if there are uncommitted changes: so that instead of going ahead with fetching changes and then suddenly aborting the rebase, we can warn user of uncommitted changes (or unclean repo state) right up front. In tests, we create a 'histedit' session to verify that also an unfinished state is detected and handled.

commit: fix unmodified message detection for the "--- >8 ----" magic We need the raw editortext to be compared with the templatetext.

commit: update test to actually modify template text We have a check for unmodified commit message (introduced by bec1a579ebc4), which should be enabled for the "--- >8 ---" magic but currently not.

pager: wrap ui._runpager As discussed at [1], ui._runpager will be the new low-level API accepting a pager command to actually run the pager. And ui.pager is the high-level API which reads config directly from self. This change is necessary for chgserver to override _runpager cleanly. [1]: www.mercurial-scm.org/pipermail/mercurial-devel/2016-December/091656.html

summary: use ui.label and join to write evolution troubles Follow-up on 7b526670f540 to avoid a convoluted loop.

log: drop unnecessary ui.note label from "trouble: " line Follow-up on f05ede08dcf7 and 6d0b1a69f98c.

revset: add a followlines(file, fromline, toline[, rev]) revset This revset returns the history of a range of lines (fromline, toline) of a file starting from `rev` or the current working directory. Added tests in test-annotate.t which already contains a reasonably complex repository.

context: add a `blockancestors(fctx, fromline, toline)` function This yields ancestors of `fctx` by only keeping changesets touching the file within specified linerange = (fromline, toline). Matching revisions are found by inspecting the result of `mdiff.allblocks()`, filtered by `mdiff.blocksinrange()`, to find out if there are blocks of type "!" within specified line range. If, at some iteration, an ancestor with an empty line range is encountered, the algorithm stops as it means that the considered block of lines actually has been introduced in the revision of this iteration. Otherwise, we finally yield the initial revision of the file as the block originates from it. When a merge changeset is encountered during ancestors lookup, we consider there's a diff in the current line range as long as there is a diff between the merge changeset and at least one of its parents (in the current line range).

mdiff: add a "blocksinrange" function to filter diff blocks by line range The function filters diff blocks as generated by mdiff.allblock function based on whether they are contained in a given line range based on the "b-side" of blocks.

run-tests: unset CHGDEBUG With CHGDEBUG, chg outputs much more stuff and the test could fail running with --chg. So unset the environment variable.

summary: add evolution "troubles" information to summary output Extend the "parent: " lines in summary with the list of evolution "troubles" in parentheses, when the parent is troubled.

summary: use the same labels as log command in "parent: " line Re-use the cmdutil._changesetlabels function introduced in 5289fd78017a to have consistent labels between the "changeset: " line in log command and the "parent: " line in summary.

templates: display evolution "troubles" in command line style

templatekw: add a "troubles" template keyword The "troubles" template keyword returns a list of evolution troubles. It is EXPERIMENTAL, as anything else related to changeset evolution. Test it in test-obsolete.t which has troubled changesets.

test: test "trouble: " line in log output with multiple troubles Follow-up on f05ede08dcf7.

cmdutil: add missing "i18n" comment about "trouble: " line Follow-up on f05ede08dcf7 per late review.

rebase: use repo.vfs.exists in 'hg summary' hook

hgweb: link to raw-file on annotation page (BC) Every other template has the "raw" link load "raw-file." However, fileannotate.tmpl's "raw" link loads "raw-annotate." This feels inconsistent and wrong. As far as I can tell, linking to the "raw annotate" view has occurred since 2006.

repair: combine two loops over changelog revisions This just saves a few lines.

repair: speed up stripping of many roots repair.strip() expects a set of root revisions to strip. It then builds the full set of descedants by walking the descandants of each. It is rare that more than a few roots get passed in, but if that happens, it will wastefully walk the changelog for each root. So let's just walk it once. I noticed this because the narrowhg extension was passing not only roots, but all the commits to strip. When there were tens of thousands of commits to strip, this resulted in quadratic behavior with that extension.

make: remove targets for building packages for ubuntu wily (end of life) Ubuntu 15.10 (Wily Werewolf) came out on October 22, 2015 and reached end of life on July 28, 2016 [1]. Users were encouraged to upgrade to 16.04 (Xenial). PPA doesn't allow new uploads targeting 15.10 anymore. [1]: https://wiki.ubuntu.com/Releases

config: add docs for ignoring all text below in the editor This is an example of how to use the new skip-from-there string for ignoring the diff in a commit message.

cmdutil: add special string that ignores rest of text Similar to git, we add a special string: HG: ------------------------ >8 ------------------------ that means anything below it is ignored in a commit message. This is helpful for integrating with third-party tools that display the

revset: drop TODO comment about sorting issue of fullreposet The bootstrapping issue was addressed at the parsing phase and we expect that fullreposet.__and__() fully complies to the smartset API, in which 'self & other' should return a result set in self's order. See also 90455e7bf543.

revset: document wdir() as an experimental function Let's resurrect the docstring since our help module can detect the EXPERIMENTAL tag and display it only if -v is specified. This patch updates the test added by 015c0d1087a3 since wdir() is now documented.

revset: categorize wdir() as very fast function The cost of wdir() should be identical to or cheaper than _intlist().

revset: make children() not look at p2 if null (issue5439) Unlike p1 = null, p2 = null denotes the revision has only one parent, which shouldn't be considered a child of the null revision. This was spotted while fixing the issue4682 and rediscovered as issue5439.

setup: pass named argument to setup_zstd The next release from upstream adds another named argument to this function. Specify arguments by name so there is no ambiguity about which argument is being passed.

merge with stable

templates-default: factor out definition of changeset labels This is redundant for normal and debug mode and prepares extension of this list that should effect both modes.

cmdutil: add support for evolution "troubles" display in changeset_printer Add a "trouble" line in changeset header along with a couple of labels on "log.changeset" line to indicate whether a changeset is troubled or not and which kind trouble occurs.

cmdutil: extract a _changesetlabels function out of changeset_printer._show() There is a common logic in changeset_printer and in the summary command for labelling a changeset. This prepares extension of changeset's labels with evolution "troubles" information that would show up in both log and summary outputs. Ultimately, both would use this function.

chg: add procutil.h This patch adds a formal header procutil.h for procutil.c, and changes Makefile to build procutil.c independently.

chg: let procutil maintain its own pagerpid Previously, chg.c maintains the pagerpid. Let's move it to procutil.c. Note: chg.c still have a pagerpid to decide whether to call attachio or not. In the future, attachio may be moved from hgc_open to hgc_runcommand, and hgc_runcommand handles both pager and attachio so we don't need to run attachio twice. And chg.c will be free of pagerpid.

chg: decouple hgclient from setuppager procutil should not depend on hgclient. This patch makes the pager handling part independent from hgclient.

chg: decouple hgclient from setupsignalhandler procutil should not depend on hgclient. This patch makes the signal handling part independent from hgclient.

chg: move signal and pager handling to a separate file In the future hgclient will deal with pager directly inside runcommand, so related signal handling stuff needs to be decoupled from chg.c. The signal handling and pager logic are coupled because we need to forward SIGPIPE when pager exits. So they are moved together, otherwise a global variable (pagerpid) is inevitable. This patch moves related functions from chg.c to procutil.c, which was marked as copied to maintain annotate history. The move is done without code modification for easy review, therefore `#include "procutil.c"` was introduced temporarily.

keepalive: rewrite readline() The old method was performing string concatenation, which is slower than collecting raw chunks in a list and joining at the end.

keepalive: remove limit argument from readline() It is unused and adds complexity.

keepalive: don't concatenate strings when reading chunked transfer Surprisingly, this didn't appear to speed up HTTP-based stream cloning on my machine. I suspect this has more to do with the fact we're using small HTTP chunks and string concatenation overhead isn't so bad. However, the reasons for this change are solid: we know string concatenation can be a performance sink.

exchange: use rich class for sorting clone bundle entries Python 3 removed the "cmp" argument from sorted(). Custom sorting in Python 3 must be implemented with the dunder comparison methods on types and/or with a "key" function. This patch converts our custom "cmp" function to a custom type. The implementation is very similar to functools.cmp_to_key(). However, cmp_to_key() doesn't exist in Python 2, so we can't use it. This was the only use of the "cmp" argument to sorted() in the code base.

chg: respect XDG_RUNTIME_DIR $XDG_RUNTIME_DIR [1] is a better place for user daemons. Let's use it and fallback to $TMPDIR. After this patch, chg will try socket paths in the following order: 1. $CHGSOCKNAME 2. $XDG_RUNTIME_DIR/chg/server 3. ${TMPDIR:-tmp}/chg$UID/server [1]: https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html

chg: make "get default sockdir" a separate method The logic to get a default socket directory will become longer in the next patch. So let's move it out.

chg: handle connect failure before errno gets overridden This patch moves the error handling logic up so that errno after connect won't be overridden.

py3: have a bytes version of shlex.split() shlex.split() only accepts unicodes on Python 3. After this patch we will be using pycompat.shlexsplit(). This patch also replaces existing occurences of shlex.split with pycompat.shlexsplit.

chg: support long socket path This patch replaces UNIX_PATH_MAX (108) with PATH_MAX (4096) so we can have long unix path.

chg: remove sockdirfd See the previous patch for the reason.

chg: let hgc_open support long path "sizeof(sun_path)" is too small. Use the chdir trick to support long socket path, like "mercurial.util.bindunixsocket". It's useful for cases where TMPDIR is long. Modern OS X rewrites TMPDIR to a long value. And we probably want to use XDG_RUNTIME_DIR [2] for Linux. The approach is a bit different from the previous plan, where we will have hgc_openat and pass cmdserveropts.sockdirfd to it. That's because the current change is easier: chg has to pass a full path to "hg" as the "--address" parameter. There is no "--address-basename" or "--address-dirfd" flags. The next patch will remove "sockdirfd". Note: It'd be nice if we can use a native "connectat" implementation. However, that's not available everywhere. Some platform (namely FreeBSD) does support it, but the implementation has bugs so it cannot be used [2]. [1]: https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html [2]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-April/082892.html

rebase: un-wrap function signature since it fits in 80 columns

py3: exclude pywatchman from test-check-py3-compat.t Exclude pywatchman from py3 test. They have already worked on Python 3 compatibility https://github.com/facebook/watchman/pull/247

py3: update test-check-py3-compat.t This part of test runs only on py3. This change was introduced by 16f4b341288d.

shelve: choose a legal shelve name when no name is passed (issue5112) Currently if our branch name contains '\' or starts with '.', shelve chooses an illegal shelve name. This behaviour is not good as it itself is choosing something which it won't accept further. We can raise errors if user passes a name which is illegal. After this patch, if '\' is contained in branch name or bookmark name, it will be replaced by '_' while choosing a shelve name and if they starts with '.', the first '.' is replaced by '_'.

shelve: add tests to ensure illegal shelve names are avoided We avoid '.' as the first letter of shelve name so that we don't create hidden file. We also avoid slashes in name so that we don't form a new directory

py3: replace sys.executable with pycompat.sysexecutable sys.executable returns unicodes on Python 3. This patch replaces occurences of sys.executable with pycompat.sysexecutable.

py3: have bytes version of sys.executable sys.executable on Python 3 returns unicodes and we want bytes. So this patch adds a new pycompat.sysexecutable which returns bytes by encoding using os.fsencode() since it is path variable.

py3: use pycompat.getcwd instead of os.getcwd

py3: use python 3 compatible variables in hgext/fsmontor/__init__.py Earlier this was left thinking that its part of pywatchman package. This patch replaces variables os.sep, sys.platform and os.envrion with their py3 compatible ones.

py3: add warnings in check-code related to py3 We have our own bytes versions of things like, getopt.getopt, os.sep, os.name, sys.executable, os.environ and few more for python 3 portability. Its better to come up with warnings if someone breaks the things which we have fixed. After this patch, check-code will warn us to use our bytes version. These checks run on mercurial/ and hgext/ and pycompat.py is excluded.

py3: replace os.getenv with pycompat.osgetenv os.getenv deals with unicodes on Python 3, so we have pycompat.osgetenv to deal with bytes. This patch replaces occurrences on os.getenv with pycompat.osgetenv

py3: have bytes version of os.getenv os.getenv() on python 3 deals with unicodes. If we want to pass bytes. we have os.getenvb() which deals with bytes. This patch adds up a pycompat.osgetenv which deals with bytes on both python 2 and 3.

url: remove unnecessary deletion of environ variables while dealing with proxy Currently we delete proxy environment variables if ui.config contains proxy values. This is unnecessary because urllib2.ProxyHandler class only reads proxy from environment it is initialised by None. But url.py never passes None, so there is no point urllib2 will take environment variables in account. This also prevents deleting environment variables which is not safe. This code was introduced while resolving Bug 2451 even it is in one of comments (sixth one) on bug that we can safely remove this part. Link to bug : https://bz.mercurial-scm.org/show_bug.cgi?id=2451

convert: add config option to control storing original revision common.commit.__init__ sets saverev=True by default. The side effect of this is that the hg sink will always set the "convert_revision" extras key to the commit being converted. This patch adds a config option to disable this behavior. While most consumers will want "convert_revision" to be a) written b) with the exact Git commit that was converted, some have use cases that prefer otherwise. In my case, I am performing significant rewrites of a Git repository *before* it is fed into `hg convert`. I have to do this because `hg convert` does not easily support the kind of transform I desire, even with extensions. (For the curious, I am "linearizing" the history of a GitHub repo by removing merge commits which add little value to the final history. It isn't easy to do this during `hg convert` because of Mercurial's file copy/rename metadata requirements.) In my scenario, my pre-convert transform stores a "convert_revision" key in the Git commit object containing the original Git commit ID. I want this original Git commit ID carried forward to Mercurial. By disabling the setting of this extra during `hg convert` and copying the value from the Git commit object, I can have the final "convert_revision" extra key contain the original Git commit ID. An added test verifies this exact scenario. This feature could likely be implemented for other VCS sources. But until someone needs the feature, I'm inclined to hold off implementing.

convert: add config option to copy extra keys from Git commits Git commit objects support storing arbitrary key-value metadata. While there is no user-facing mechanism in Git to record these values, some tools do record data here. Currently, `hg convert` only handles the "author," "committer," and "parent" keys in Git commit objects. All other keys are ignored. This means that any custom keys are lost when converting Git repos to Mercurial. This patch implements support for copying a whitelist of extra keys from Git commit objects to the "extras" dict of the destination. As the added tests demonstate, this allows extra metadata to be preserved during the conversion process. This patch stops short of converting all metadata to "extras." We could potentially implement this via `convert.git.extrakeys=*` or similar. But copying everything by default is a bit dangerous because if Git adds new keys to commit objects, we could find ourselves copying things that shouldn't be copied! This patch also assumes the source key is the same as the destination key. We could implement support for prefixing the output key to distinguish it as coming from Git. But until this feature is needed, I'm inclined to hold off implementing it.

convert: don't use {} as default argument value This is a common Python gotcha. I'm kinda surprised we don't have a check-code to detect this :/

documentation: better censor flag documentation

tests: make test-convert-git.t reproducible For reasons I can't explain, Git's copy detection code was identifying different source files on OS X and (presumably) Solaris versus Linux (which the test was originally authored against). This was causing unstable test output. Changing the test to use a non-ambiguous source file appears to make the test stable. The test was introduced recently in ea3540e66fd8.

fsmonitor: refresh pywatchman to upstream Update to upstream to version c77452. The refresh includes fixes to improve windows compatibility. There is a minor update to 'test-check-py3-compat.t' as c77452 no longer have the py3 compatibility issues the previous version had. # no-check-commit

tests: exclude bundled pywatchman from test-check-pyflakes The code under pywatchman used to be excluded from test-check-code and test-check-pyflakes through the magic string "no-check-code". Now that test-check-code excludes the pywatchman directory entirely, the directory should also be excluded from test-check-pyflakes.

color: load 'colortable' from extension using an 'extraloader' Now that we have the '_style' dictionary in core, we can use the clean and standard 'extraloader' mechanism to load extension's 'colortable'. color.loadcolortable

color: move hgext.color._styles to mercurial.color.style This is small first step to start moving the color infrastructure into core. The current code of the color extensions is full of strange and debatable things, we'll clean it up in the process as having things into core help the cleaning. Moving _style was the simplest sensible move that is possible. It will also help cleaning up the extension setup process in a later changesets.

color: drop use of the 'global' keyword for '_style' Using 'global' is usually a bad sign. Here it is used so that one can empty the content of a dict at the global scope. We '_style.clear()' and drop the global.

color: drop use of the 'global' keyword for _terminfo_params Using 'global' is usually a bad sign. Here it is used so that one can empty the content of a dict at the global scope. We '_terminfo_params.clear()' and drop the global.

fsmonitor: fix exception message scraping pywatchman.CommandError formats its error message such that 'unable to resolve root' is not a prefix. This change fixes that by instead just searching for it as a substring.

tests: add magic string to bypass check-commit Allow bypassing test-check-commit runs by specifying '# no-check-commit' in the description. This should be avoided but is useful for upstream imports such as pywatchman which will cause check-code to fail otherwise.

convert: config option for git rename limit By default, Git applies rename and copy detection to 400 files. The diff.renamelimit config option and -l argument to diff commands can override this. As part of converting some repositories in the wild, I was hitting the default limit. Unfortunately, the warnings that Git prints in this scenario are swallowed because the process running functionality in common.py redirects stderr to /dev/null by default. This seems like a bug, but a bug for another day. This commit establishes a config option to send the rename limit through to `git diff-tree`. The added tests demonstrate a too-low rename limit doesn't result in copy metadata being recorded.

chgserver: backout changeset dfb19aed409e (per discussion) On Wed, 21 Dec 2016 15:39:05 +0000, Jun Wu wrote: > Actually, patch 1 is unnecessary if we go with the "ui._runpager" approach. > Maybe someone can drop it without adding too many markers.

chgserver: override runcommand Next patches will customize chgserver's runcommand. So let's override it. The docstring is temporarily missing and will be filled later.