rebase: delete the comment which was not following "do not eat my data" Differential Revision: https://phab.mercurial-scm.org/D3756

tags: unblock log-like template keywords and functions It checks if ctx will be used in template since loading ctx per revision could take extra 10-100msec in total depending on the number of tags.

formatter: provide hint of context keys required by template This allows us to create ctx objects only if necessary. I tried another idea which is to populate ctx from 'repo' and 'node' value on demand. It worked, but seemed unnecessarily complicated. So I chose a simpler one. The datafields argument is a space-separated string for consistency with fm.write() API.

templatefuncs: declare resource requirements for future use

templatefuncs: minimize resource requirements

merge with stable

split: preserve phase of commit that is being split With this change, hg split will preserve the phase of the commit that is being split, ignoring the phases.new-commit setting. Previously, we would use whatever phases.new-commit was set to (unless our parent was secret, then we would be secret even if phases.new-commit=draft). Now, splitting a draft commit with phases.new-commit=secret does not cause the new commits to become secret, and splitting a secret commit with phases.new-commit=draft and a draft parent does not cause the new commits to become draft. Test cases and commit message taken from Kyle Lippincott's D2016 (thanks!). Differential Revision: https://phab.mercurial-scm.org/D3819

scmutil: make cleanupnodes optionally also fix the phase We have had multiple bugs where the phase wasn't correctly carried forward to a rewritten changeset (for example: phabricator, split, evolve, fix). Handling the phase update in cleanupnodes() makes it less likely to happen again, especially once we have made it fix the phase by default (perhaps in the next release cycle). This patch also updates all applicable callers so we get some testing of it. Note that rebase and histedit can't be fixed yet because they call cleanupnodes() only at the end and the phase may have been changed by the user when the rebase/histedit was interrupted (due to merge conflicts). I think we should make them write one commit at a time (as it already does), along with associated obsmarkers, bookmark moves, etc. When that's done, we can switch them over to cleanupnodes(). Differential Revision: https://phab.mercurial-scm.org/D3818

tests: add test of uncommit with default phase as secret We didn't seem to have any test checking that uncommitting a draft commit with phase.new-phase=secret preserved the draft phase. Differential Revision: https://phab.mercurial-scm.org/D3817

grep: add --diff flag Adds a diff flag, which works exactly same as all, in fact since --all searches diffs, there diff is a better name for it. The all flag is still here for backward compatibility reasons. Some major tests for all has been picked and added for diff. Differential Revision: https://phab.mercurial-scm.org/D3763

fix: include cleanupnodes() in transaction As pointed out by Yuya, we need a transaction to make sure the state before the call to cleanupnodes() is not observable. Differential Revision: https://phab.mercurial-scm.org/D3823

progress: enforce use of complete() on the helper class complete() is preferred over update(None), so let's enforce that. Differential Revision: https://phab.mercurial-scm.org/D3822

progress: extract function for closing topic progress(None) had a completely different implementation from the progress(<not None>) implementation. It very much feels like it should be a separate method, so this patch makes it so. That also makes it clear that only the topic parameter matters when closing a topic (e.g. "total" does not matter). Differential Revision: https://phab.mercurial-scm.org/D3821

progress: use context manager for lock Differential Revision: https://phab.mercurial-scm.org/D3820

configitem: reorder items in the 'server' section Keeping things alphabetically sorted.

test-lfs: add coverage for the binary() fileset This ensures that the blobs don't need to be present to be filtered properly.

fileset: use filectx.isbinary() to filter out binaries in eol() Since LFS stores the binary attribute in the pointer file, this means that the file doesn't need to be downloaded in order to be skipped. This function also catches an IOError if the data can't be loaded in the non-LFS case. I wonder if it's worth storing the unix/dos attributes in the pointer file as well, though I'd expect LFS files to be binary most of the time.

crecord: re-center display in interactive curses commit on pageup/down A long-standing issue in the crecord (interactive curses commit interface) is that using PageUp/Down to move along longer-than current screen size chunks would "lose" the cursor and not properly re-center. There has been self.recenterdisplayedarea() to do that, but it has not been in use for some reason. Add calls to the appropriate uparrowshiftevent() and downarrowshiftevent() methods to fix this.

tests: in test-split.t, save a "clean" copy of pre-split repo for later use Differential Revision: https://phab.mercurial-scm.org/D2015

tests: use progress helper With this commit, the only in-tree caller of ui.progress() is scmutil.progress(). That means that we could deprecate it. It also means that we can considering inlining it in scmutil.progress. Differential Revision: https://phab.mercurial-scm.org/D3812

changegroup: use progress helper Although it looks like this code was micro-optimized, I could not measure any slow-down. Differential Revision: https://phab.mercurial-scm.org/D3811

synthrepo: close progress topics Differential Revision: https://phab.mercurial-scm.org/D3810

synthrepo: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3809

largefiles: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3808

convert: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3807

lfs: use progess helper Differential Revision: https://phab.mercurial-scm.org/D3806

relink: use progress helper This doesn't use progress.increment() because progress output is skipped for some positions (so we may end up calling "update(0), update(2), update(7)", or similar). Differential Revision: https://phab.mercurial-scm.org/D3805

patchbomb: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3804

patchbomb: don't close unused progress topic The "writing" topic has not been used since 1830d0cc4bc1 (patchbomb: minor refactoring of mbox functionality, preparing for move, 2011-11-23). Differential Revision: https://phab.mercurial-scm.org/D3803

churn: use progess helper Differential Revision: https://phab.mercurial-scm.org/D3802

treediscovery: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3801

upgrade: use progress helper A minor side-effect is that we no longer print the progress at 0 (and that we don't re-print it at its current value when starting the next manifest/file). Differential Revision: https://phab.mercurial-scm.org/D3800

upgrade: close progress after each revlog IIUC, one is supposed to close each progress topic before strarting a new one. Otherwise the topics are considered nested, which we don't want here. Differential Revision: https://phab.mercurial-scm.org/D3799

verify: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3798

verify: use progress helper for subdirectory progress I also reworded a variable to make it clearer that it's only used for subdirectories. Differential Revision: https://phab.mercurial-scm.org/D3797

similar: use progress helper A side-effect is that progress is now reported as 1 *before* we start checking the first file. That seems to be how we do it in most places. Also, the right topic is now closed. Differential Revision: https://phab.mercurial-scm.org/D3796

repair: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3795

httpconnection: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3794

py3: byte-stringify literals in contrib/phabricator.py as example Transformed by contrib/byteify-strings.py and adjusted exceeded lines manually. Some of b''s would be wrong as the phabriactor extension has to work with JSON data. # skip-blame just many b prefixes

byteify-strings: remove superfluous "if True" block

byteify-strings: try to preserve column alignment

byteify-strings: do not rewrite system string literals to u'' It would make things worse on Python 2 because unicode processing is generally slower than byte string. We should just leave system strings unmodified.

byteify-strings: do not rewrite iteritems() and itervalues() by default We can't do that automatically due to performance concerns.

byteify-strings: drop import-line hack This is ugly, and valid only for Python 3. We'll need to find a different way if we want to get rid of the code transformer at all.

byteify-strings: add --inplace option to write back result

byteify-strings: add basic command interface

byteify-strings: fork py3 code transformer to make it a standalone command I'm thinking of making a one-off s/''/b''/g change for overall codebase to make linter happy. We could do that without maintaining the script, but I think it will be somewhat useful for extension authors. So it is in contrib.

highlight: get around tmpl.load() which now returns a parsed tree Broken at e637dc0b3b1f, "templater: parse template string to tree by templater class."

changegroup: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3783

archival: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3782

copystore: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3781

subrepo: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3780

histedit: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3779

rebase: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3778

debugbuilddag: use context manager for progress, locks, transaction I minor side-effect is that .hg/localtags is now written before the transaction commits. Differential Revision: https://phab.mercurial-scm.org/D3777

debugbuilddag: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3776

progress: make the progress helper a context manager This lets us simplify the use site in streamclone. Differential Revision: https://phab.mercurial-scm.org/D3775

progress: hide update(None) in a new complete() method update(None) seemed a bit cryptic. Differential Revision: https://phab.mercurial-scm.org/D3774

rebase: add dry-run functionality For now, it gives stats about rebase would be successful or hit a conflict. Remaining work is to improve the output and adding verbose mode where will show the diff of conflicting files if we hit any. Differential Revision: https://phab.mercurial-scm.org/D3757

py3: add `and None` to suppress return values of .write() calls .write() calls don't return anything on Python2, so we need to make sure we suppress that on py3 too. This makes the test pass on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3793

py3: slice over bytes to prevent getting ascii values Differential Revision: https://phab.mercurial-scm.org/D3792

py3: use pycompat.maplist() instead of map() in hgext/transplant.py map() returns a map-object on Python3 instead of a list. Differential Revision: https://phab.mercurial-scm.org/D3791

py3: add 4 new passing tests to whitelist Differential Revision: https://phab.mercurial-scm.org/D3790

py3: make tests/test-impexp-branch.t compatible with Python 3 This patch adds some missig b'' prefixes and add suppress the output of write() calls. Differential Revision: https://phab.mercurial-scm.org/D3789

py3: convert error instances to bytes using pycompat.bytestr() Differential Revision: https://phab.mercurial-scm.org/D3788

py3: encode sys.argv to bytes using .encode() Differential Revision: https://phab.mercurial-scm.org/D3787

py3: use '%d' for os.stat_result.st_nlink instead of '%s' Differential Revision: https://phab.mercurial-scm.org/D3786

py3: suppress the output of .write() calls in tests/test-revlog-v2.t This makes the test pass on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3785

py3: add couple of missing b'' prefixes in tests/test-pager-legacy.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3784

import: use context manager for lock, dirstateguard, transaction A tiny side-effect is that the transaction is now closed after saving the commit message. Differential Revision: https://phab.mercurial-scm.org/D3748

streamclone: update progress later to avoid passing None to util.bytecount Differential Revision: https://phab.mercurial-scm.org/D3773

highlight: adjust to attribute being private Differential Revision: https://phab.mercurial-scm.org/D3772

annotate: automatically populate fields referenced from template If '{line_number}' is in the template, we'll probably need it. No --line-number option should be required in such cases.

annotate: reverse mapping between option name and field name This makes the next patch slightly simpler.

formatter: provide hint of referenced field names I don't like the function name, but I call it datahint() for consistency with another function I'll add later, fm.contexthint().

templater: add function to look up symbols used in template Formatter can use this information to enable slow paths such as loading ctx object only when necessary.

templater: parse template string to tree by templater class The parsed tree could be cached, but it isn't for now. We can add a cache later if that matters.

templater: drop extension point of engine classes (API) I don't think this would ever be used by third-party extensions, as we've heavily changed both the templater internals and the syntax since then. The main reason of removing this API is that I want to move the parsing function from the engine to the templater class so that we can peek keywords and functions used in a user template. This change also removes reference cycle between the templater and the engine.

templater: make it clearer that parsing doesn't cause recursion Only compileexp() may recurse into _load().

templater: mark most attributes as private

setdiscovery: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3770

streamclone: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3769

similar: use progress helper Note that a functional change here is that we now show the progress position *before* we start working on the item. This is consistent with how we do it elsewhere. Differential Revision: https://phab.mercurial-scm.org/D3768

remove: use progress helper Differential Revision: https://phab.mercurial-scm.org/D3767

changegroup: use progress helper in apply() (API) Differential Revision: https://phab.mercurial-scm.org/D3766

progress: create helper class for incrementing progress When using ui.progress(), there's a clear pattern that is followed: * Pass the same topic and unit * Usually pass the same total * Call with pos=None to close the progress bar * Often keep track of the current position and increment it This patch creates a simple helper class for this. I'll probably make it implement the context manager protocol later (calling update(None) on __exit__). Progress is used in low-level modules like changegroup, so I also exposed it via a method on the ui object. Perhaps the class itself should also live in ui.py? This patch also makes merge.oy use it to show that it works. Differential Revision: https://phab.mercurial-scm.org/D3765

extensions: use context manger for open()

py3: open extension source in binary mode to read docstring as bytes

grep: adds allfiles mode Adds an allfiles flag that lets you grep on all files in the revision and not just the one that were modified in that changeset. This would work on a single revision and get all the files that were there in that revision. So it's like grepping on a previous state. Using this with wdir() :: `hg grep -r "wdir()" --allfiles` is what the default behavior is desired for grep. Support for multiple revisions to be added later. Differential Revision: https://phab.mercurial-scm.org/D3728

morestatus: remove some extra spaces The information about unfinished states in `hg status -v` had a lot of spaces which are not required and feels weird. Let's limit the spacing to four spaces. Differential Revision: https://phab.mercurial-scm.org/D3730

graph: improve graph output by using Unicode characters This extension beautifies log -G output by using Unicode characters. A terminal with UTF-8 support and a monospace Unicode font are required. Differential Revision: https://phab.mercurial-scm.org/D3665

py3: whitelist another 5 passing tests thanks to the ratchet Differential Revision: https://phab.mercurial-scm.org/D3762

shelve: wider check for successful abort in test The test was previously focusing on checking that the "sub-rebase" aborted properly. Since rebase is an implementation details we should also check for other messages. Differential Revision: https://phab.mercurial-scm.org/D3686

shelve: use full hash in tests Using revision number is fragile. Moving to full hash to help with further development. Differential Revision: https://phab.mercurial-scm.org/D3685

phabricator: preserve the phase when amending in the Differential fields I have no idea if it's better to change scmutil.cleanupnodes() so that it has the option to either apply a specific phase (e.g. for various --secret switches) or carry over the phase of the old node. The benefit would be that the caller doesn't have to remember to do this. The con is maybe inefficiency? I wrote this up as issue5918. I'm leaving that open since Yuya flagged it as an API bug. Since most other callers already do this, it's the simplest fix. (It's not obvious that `split`, `fix` and `rebase` are doing this, but there is test coverage for `fix` and `rebase`, and experimenting with `split` shows it does the right thing.)

py3: ditch email.parser.BytesParser which appears to be plain crap As I said before, BytesParser is a thin wrapper over the unicode Parser, and it's too thin to return bytes back. Today, I found it does normalize newline characters to '\n's thanks to the careless use of TextIOWrapper. So, this patch replaces BytesParser with Parser + TextIOWrapper, and fix newline handling. Since I don't know what's the least bad encoding strategy here, I just copied it from BytesParser. I've moved new parse() function from pycompat, as it is no longer a trivial wrapper.