packaging: move builddeb into contrib/packaging/ Differential Revision: https://phab.mercurial-scm.org/D3544

packaging: move packagelib.sh into contrib/packaging/ This required some minor path adjustments in scripts referencing this path. Differential Revision: https://phab.mercurial-scm.org/D3543

packaging: move some docker scripts into contrib/packaging/ This required some minor path adjustments in the scripts. Differential Revision: https://phab.mercurial-scm.org/D3542

packaging: move most of contrib/docker to contrib/packaging/docker contrib/ is kind of a dumping ground for all kinds of random things. Many of the files in contrib/ are related to packaging. But sorting out how everything is related is a lot of effort since it is all mixed together. This commit starts the process of refactoring everything related to packaging so it all lives under one roof. Most content from contrib/docker is simply moved to contrib/packaging/docker and all references are updated. Differential Revision: https://phab.mercurial-scm.org/D3541

setup: reformat options argument This makes things a bit more easier to manage IMO. Differential Revision: https://phab.mercurial-scm.org/D3540

contrib: remove fixpax.py This was used as part of bdist_mpkg. And we stopped using bdist_mpkg after db5084d27df5. Differential Revision: https://phab.mercurial-scm.org/D3539

revlog: handle errors from index_node() in nt_insert() and index_slice_del() Same idea as in a9d9802d577e (revlog: don't say "not found" on internal error, 2018-05-04). Differential Revision: https://phab.mercurial-scm.org/D3558

help: add new deprecated topic Add a deprecated topic which lists most of the commands and options that are not recommended anymore.

phabricator: split auth.url into the standard auth.schemes and auth.prefix It seems better to reuse the existing function to find the proper [auth] block, even if not all of the possible settings may be of interest. The other callers of readauthforuri() make a trip through the password database to fetch the user from the URI. But in the little experimenting I did here, the username always came back as None. Since readauthforuri() wants it to make sure that user@prefix matches user@url, it seems that parsing the URL and pulling out the user component should be equivalent.

phabricator: migrate [phabricator.auth] to [auth] This resurrects the warning mechanism removed in 20a4543e9a2b. I'm not worried about the copy/paste of the code for a brief transitional period.

dispatch: unify handling of None returned by a command function A command function may return None in place of 0 just for convenience, but dispatch() doesn't need to inherit that property. This patch makes it be friendly to callers.

dispatch: minor code refactor This hopefully makes things slightly easier to read. Differential Revision: https://phab.mercurial-scm.org/D3537

hgweb: drop unused argument 'tmpl' from webutil.compare()

hgweb: convert {comparison} to a mappinggenerator with named template No bare generator.

hgweb: rename 'context' argument of webutil.compare() to avoid name conflicts The meaning of 'context' depends on context. Here it is the number of the context lines in unified diff.

hgweb: convert comparison {lines} to a mappinggenerator with named template Bare generator can't be restarted.

hgweb: move getblock() closure out of compare() (Please use 'hg diff -w' to get readable diff from this patch.)

hgweb: move compline() closure out of compare()

hgweb: convert {diff} to a mappinggenerator with named template No more bare generator. Fortunately, this one is associated with a single template, so it can be a mappinggenerator.

hgweb: use template context to render {lines} of {diff} This is a preferred way to process nested templates.

hgweb: wrap {lines} of {diff} with mappedgenerator This can't be a mappinggenerator as it switches the templates conditionally.

hgweb: move prettyprintlines() closure out of diffs() This will be wrapped with mappedgenerator.

remotenames: enable the storage config option if extension is enabled Before this patch, the config option to store remotenames was set to False by default and remotenames extension does not set it True. So if you enable remotenames extension without setting 'experimental.remotenames=True', you won't get the remotenames. This patch makes remotenames enable the config option if then extension is enabled which makes sense. Differential Revision: https://phab.mercurial-scm.org/D3520

tests: clone the repo over ssh in tests/test-logexchange.t Cloning a local repo in tests, and normally too does not perform the actual clone, rather it copies the file contents. This makes remotenames information not being transferred. Let's clone over ssh so that we can test remotenames correctly. Differential Revision: https://phab.mercurial-scm.org/D3519

remotenames: check the remotepath with url containing user information too Current logic to match a remotepatch to a user defined path first removes the authentication information from the url and then tries to match it. However this is not true every time. If we clone a repo using ssh, the default path contains the user information too. Differential Revision: https://phab.mercurial-scm.org/D3518

shortest: avoid magic number "41" As suggested by Yuya. Feel free to fold into D3502. Differential Revision: https://phab.mercurial-scm.org/D3531

revlog: handle error from node lookup I missed this in D3499. Thanks to Yuya for spotting it. Differential Revision: https://phab.mercurial-scm.org/D3530

debugextensions: process extensions loaded from the local repository too

phabricator: drop support for the deprecated `phabricator.token` config

help: load module doc of disabled extension in extensions.disabledcmd() This helps rewriting _finddisablecmd() to not load the module.

extensions: extract closure that looks for commands from disabled module I'll rewrite this function to not load disabled extensions.

extensions: drop dead code trying to exclude deprecated disabled commands Here 'ext' is a tuple, which means ext.__doc__ describes about the Python tuple type. Fixing this would break 'hg help glog', which currently shows some nice explanation, so this patch just removes the dead code. $ hg help glog 'glog' is provided by the following extension: graphlog command to view revision graphs from a shell (DEPRECATED) (use 'hg help extensions' for information on enabling extensions)

util: drop remainder of dateutil/procutil aliases (API)

shortest: move revnum-disambiguation out of revlog I want to be able to change how we disambiguate and I think having revlog.shortest() worry only about finding a prefix that's unambiguous among nodeids makes more sense. This slows down `hg log -T '{shortest(node,1)}\n'` from 4.0s to 4.1s. Differential Revision: https://phab.mercurial-scm.org/D3502

shortest: remove unnecessary check for revnum in isvalid() We now always disambiguating against revnums after we've found an otherwise valid prefix, so we can safely remove the early isrev() check in isvalid(). Differential Revision: https://phab.mercurial-scm.org/D3501

shortest: make pure code also disambigute against revnums at end This makes the pure code more similar to the native code in that it first finds a prefix that's unambiguous among nodeids and then adds hex digits until it no longer looks like a revnum. It will allow us to even better separate the disambiguation with revnums in a later patch. With this patch `hg log -r 0::50 -T '{shortest(node,1)}'` with no native code goes from 25s to 43s. It wasn't exactly usable to begin with, so I don't feel too bad about it. Differential Revision: https://phab.mercurial-scm.org/D3500

revlog: use node tree (native code) for shortest() calculation I want to rewrite revlog.shortest() to disambiguate only among hex nodeids and then disambiguate the result with revnums at a higher level (in scmutil). However, that would slow down `hg log -T '{shortest(node,1)}\n'` from 5.0s to 6.8s, which I wasn't sure would be acceptable. So this patch makes revlog.shortest() use the node tree for finding the length of the shortest prefix that's unambiguous among nodeids. Once that has been found, it makes it longer until it is also not ambiguous with a revnum. This speeds up `hg log -T '{shortest(node,1)}\n'` from 5.0s to 4.0s. Differential Revision: https://phab.mercurial-scm.org/D3499

shelve: reduce scope of merge tool config override The config override seems to have a much greater scope than it needed to. I *think* it's only relevant in the while merging files. The rebase step also cares about the merge tool, but we seem to be explicitly passing it to rebase (around line 755). Differential Revision: https://phab.mercurial-scm.org/D3517

util: drop _deprecatedfunc() It was only needed for the previously removed forwarding.

util: drop deprecated forwarding to procutil (API)

tests: migrate demandimport.py away from deprecated `util` module symbols

templatekw: drop deprecated showlist() and showdict() (API)

util: drop deprecated forwarding to dateutil (API)

util: drop deprecated forwarding to stringutil (API)

scmutil: remove deprecated revpairnodes method (API)

merge: drop support for using updateresults as tuples (API)

cmdutil: drop deprecated log helper methods (API)

cmdutil: drop deprecated precursor of registrar.command (API)

cmdutil: remove deprecated _revertprefetch code (API)

tests: fix deprecation warning in test-url.py

hgweb: remove unused argument 'tmpl' from listfilediffs()

hgweb: use template context to render {files} of changelist entries This is a preferred way to process nested templates.

hgweb: wrap {files} of changelist entries with mappedgenerator This also switches the associated templates conditionally, which can't be a mappinggenerator.

hgweb: use template context to render {files} of changesetentry() This is a preferred way to process nested templates.

hgweb: wrap {files} of changesetentry() with mappedgenerator This can't be easily adapted to mappinggenerator because the associated templates are switched per item.

hgweb: extract generator of {files} from changesetentry() This will be wrapped with mappedgenerator.

templatefilters: document the json filter I don't know the exact reason why it was undocumented, but that would be because unicode is difficult for humans. Since the json filter can now get localstr back to UTF-8 as long as it is unprocessed (i.e. {desc|json} works, but {desc|firstline|json} doesn't), it seems good enough to advertise the json filter.

encoding: introduce tagging type for non-lossy non-ASCII string This fixes the weird behavior of toutf8b(), which would convert a local string back to UTF-8 *only if* it was lossy in the system encoding. Before b7b26e54e37a "encoding: avoid localstr when a string can be encoded losslessly (issue2763)", all local strings were wrapped by the localstr class. I think this would justify the round-trip behavior of toutf8b(). ASCII strings are special-cased, so the cost of wrapping with safelocalstr is negligible. (with mercurial repo) $ export HGRCPATH=/dev/null HGPLAIN= HGENCODING=latin-1 $ hg log --time --config experimental.evolution=all > /dev/null (original) time: real 11.340 secs (user 11.290+0.000 sys 0.060+0.000) time: real 11.390 secs (user 11.300+0.000 sys 0.080+0.000) time: real 11.430 secs (user 11.360+0.000 sys 0.070+0.000) (this patch) time: real 11.200 secs (user 11.100+0.000 sys 0.100+0.000) time: real 11.370 secs (user 11.300+0.000 sys 0.070+0.000) time: real 11.190 secs (user 11.130+0.000 sys 0.060+0.000)

encoding: fix toutf8b() to resurrect lossy characters even if "\xed" in it If 's' is a localstr, 's._utf8' must be returned to get the original UTF-8 sequence back. Because of this, it was totally wrong to test if '"\xed" not in s', which should be either '"\xed" not in s._utf8' or just omitted. This patch moves the localstr handling to top as the validity of 's._utf8' should be pre-checked by encoding.tolocal().

sshserver: redirect stdin/stdout early and use duplicated streams This is what we achieved with hook.redirect(True) plus ui.fout = ui.ferr. The hook.redirect() function can't be completely removed yet since hgweb still depends on it. I'm not sure if it is necessary for WSGI servers. CGI needs it, but does WSGI communicate over stdin/stdout channels?

sshserver: do setbinary() by caller (API) In most cases, stdio should be set to binary mode by the dispatcher, so the sshserver does not have to take care of that. The only exception was hg-ssh, which is fixed by this patch. .. api:: ``sshserver()`` no longer sets stdin and stdout to binary mode.

test-ssh: add some flush() to make output deterministic We shouldn't rely on buffering mode/state of file handles.

stringutil: flip the default of pprint() to bprefix=False If we use pprint() as a drop-in replacement for repr(), bprefix=False is more appropriate. Let's make it the default to remove bprefix=False noise.

stringutil: make pprint() forward uninteresting object to b'%r' We appear to start using pprint() as a replacement for repr(), so it's probably safer to support any Python objects instead of complaining about that.

tests: use stringutil.pprint instead of custom dumper in test-hgweb-auth.py Differential Revision: https://phab.mercurial-scm.org/D3529

tests: port test-simplekeyvaluefile.py to Python 3 # skip-blame just b prefixes Differential Revision: https://phab.mercurial-scm.org/D3528

tests: port test-extensions-wrapfunction to Python 3 Differential Revision: https://phab.mercurial-scm.org/D3527

tests: port test-ui-config to Python 3 Slight test output changes to make life easier. Differential Revision: https://phab.mercurial-scm.org/D3526

config: fix py3 backslash escaping bug in parser caught by tests Differential Revision: https://phab.mercurial-scm.org/D3525

tests: port test-symlink-os-yes-fs-no.py to Python 3 # skip-blame just bytes/str changes Differential Revision: https://phab.mercurial-scm.org/D3524

posix: use inst.errno instead of inst[0] on OSError instances Differential Revision: https://phab.mercurial-scm.org/D3523

tests: port test-filelog.py to Python 3 # skip-blame just some bytes prefixes Differential Revision: https://phab.mercurial-scm.org/D3522

stringutil: teach pprint about tuples Differential Revision: https://phab.mercurial-scm.org/D3521

templatefuncs: show hint if extdata source is evaluated to empty (issue5843)

revlog: use literal -1 instead of variable that always has that value We were setting "ntrev" to "rev", but "rev" was always -1 at the end of the loop, so it's clearer to use a literal -1. Differential Revision: https://phab.mercurial-scm.org/D3498

revlog: extract function for fully populating the radix tree This code is currently used for partialmatch(), but I want to reuse it when I implement shortest() in native code. Differential Revision: https://phab.mercurial-scm.org/D3497

tests: port test-lfs-pointer.py to Python 3 The weird bit here is having to import /something/ from Mercurial before we touch hgext.* so that the module loader gets initialized. That's probably a bug we should explore at some point? Differential Revision: https://phab.mercurial-scm.org/D3514

tests: port test-ui-verbosity.py to Python 3 # skip-blame just some bytes/string wrangling Differential Revision: https://phab.mercurial-scm.org/D3515

lfs: stabilize error message values for Python 2 and 3 Differential Revision: https://phab.mercurial-scm.org/D3513

tests: port test-ui-color.py to Python 3 As in some other tests, I like confirming that we're dealing with bytes here, so the b prefix is now in the expected output on both 2 and 3. Differential Revision: https://phab.mercurial-scm.org/D3512

tests: port test-dispatch.py to Python 3 Differential Revision: https://phab.mercurial-scm.org/D3511

tests: port test-wireproto.py to Python 3 A little bit of output churn because we now get b'' prefixes on output in both 2 and 3, but for this test I'm more comfortable knowing that we're using bytestrs everywhere. Differential Revision: https://phab.mercurial-scm.org/D3510

tests: port test-context.py to Python 3 This is a weird "minimally invasive" port. I think there's room to do better here, but I'm also not sure how often we'll touch this code... Differential Revision: https://phab.mercurial-scm.org/D3509

scmutil: fix __repr__ of status tuple We should probably start giving some thought to migrating the status tuple to attrs, but not now. Differential Revision: https://phab.mercurial-scm.org/D3508

update: print warning about hidden changeset after update When an attempt to update to a hidden changeset fails because the working copy is dirty, you may get a message like this: updating to a hidden changeset 343f6de32686 (hidden revision '343f6de32686' was rewritten as: 4ab941244072) abort: conflicting changes (commit or update --clean to discard changes) It's easy to miss the real error here. This patch moves the warning about the hidden changeset to after the update has happened. It changes the verb tense accordingly (and drops the "a" that I think it sounds better without). Of course, this means that the commit isn't actually hidden anymore when the message is printed. I think that's fine. Differential Revision: https://phab.mercurial-scm.org/D3479

py3: whitelist three passing cvs convert tests Differential Revision: https://phab.mercurial-scm.org/D3490

cvsps: convert encoding name to sysstr Differential Revision: https://phab.mercurial-scm.org/D3489

tweakdefaults: remove "bisect" from commands.status.skipstates If you forget to clear bisect state after you're done bisecting (as I've done twice in the last week or so), the next time you run `hg bisect --good/--bad`, it's going to tell you "The first bad revision is:" etc. It's probably not obvious to new users what's going on, and having the verbose output about an ongoing bisection will probably help them, so let's turn it back on by default. Differential Revision: https://phab.mercurial-scm.org/D3516

hgweb: wrap {instabilities} by hybridlist() This is a list of simple values, which can be a hybrid list.

hgweb: wrap {whyunstable} with mappinggenerator This is also a generator of mappings, which needs a wrapper.

hgweb: wrap {succsandmarkers} with mappinggenerator This is also a generator of mappings, which needs a wrapper.

hgweb: wrap {branches} and {entries} of branches with mappinggenerator Bare generator of mappings shouldn't be put in a template mapping because its type can't be determined without consuming it.

hgweb: drop tmpl argument from webutil.showtag() and showbookmark() It's replaced by a context argument passed to a mappinggenerator.

hgweb: wrap {changelogtag}, {changesettag}, and {changesetbookmark} These can't be hybrid lists as they've associated with named template, 't1'.

hgweb: drop useless **args from webutil.showtag() and showbookmark() Callers never pass excessive arguments to these functions.

hgweb: reuse graph node-related functions from templates The difference between templatekw.getgraphnode() and webutil.getgraphnode() is that the latter is not limited to 1 character.

templates: split getgraphnode() body into two functions getgraphnodecurrent() is checking if the node is currently checked out and getgraphnodesymbol() is checking properties that have more to do with the stored data and the DAG.

hgweb: wrap {branch} and {changesetbranch} by hybridlist() This is also a 0/1-length list of a simple value, can be a hybrid list. Appears that we have many {branch} variants.

hgweb: wrap {inbranch} by hybridlist() This is also a 0/1-length list of a simple value, can be a hybrid list.

hgweb: wrap {branches} by hybridlist() This is a 0/1-length list of a simple value, can be a hybrid list.

hgweb: wrap {bookmarks} by hybridlist() This is also a list of simple values.

hgweb: wrap {tags} by hybridlist() This one is a list of simple values, which can be a hybrid list.

hgweb: wrap {rename} with mappinglist No bare list of mappings should be put in a template mapping.

tests: fix test-status-inprocess.py on Python 3 The same print() hack as test-filecache.py. Differential Revision: https://phab.mercurial-scm.org/D3507

tests: port test-filecache.py to Python 3 Only remarkable bit is my wrapper around print(), which I regret a little, but not enough to go back and try to do something cleaner. Differential Revision: https://phab.mercurial-scm.org/D3506

tests: port test-cbor.py to Python 3 I suspect this b''.join() was a remnant of an earlier iteration of this code, as it was building a string from a string. Differential Revision: https://phab.mercurial-scm.org/D3505

cborutil: port to Python 3 The only problem lurking in here was sorts of mismatched types. The sorts are only for output stability in our tests (sigh), so we just build a phony sort key using the __name__ of types so that we only compare like types against each other. By pure luck, my awful sort key matches the behavior we get "for free" in Python 2, so no test output changes. Differential Revision: https://phab.mercurial-scm.org/D3504

lock: pass sysstr to warnings module This makes test-lock.py slightly less broken, but it's still pretty far from passing. Differential Revision: https://phab.mercurial-scm.org/D3503

tests: port test-revlog-raw.py to Python 3 # skip-blame just b prefixes Differential Revision: https://phab.mercurial-scm.org/D3496

tests: migrate test-wsgirequest.py to Python 3 # skip-blame adding and removing b prefixes Differential Revision: https://phab.mercurial-scm.org/D3495

tests: port test-parseindex2.py to Python 3 # skip-blame just b prefixes and int() instead of long() Differential Revision: https://phab.mercurial-scm.org/D3494

tests: prefer string concatenation with () instead of \ in parseindex2 tests Differential Revision: https://phab.mercurial-scm.org/D3493

tests: port test-simplemerge.py to Python 3 # skip-blame just b prefixes Differential Revision: https://phab.mercurial-scm.org/D3492

tests: replace use of assertEquals with assertEqual in test-simplemerge.py The former is deprecated. No functionality change. # skip-blame just removing an ess and some whitespace Differential Revision: https://phab.mercurial-scm.org/D3491

tests: b prefixes on some inline python in test-convert-cvs.t # skip-blame just b prefixes Differential Revision: https://phab.mercurial-scm.org/D3488

cvsps: wrap bytes in bytestr before %r-ing it Differential Revision: https://phab.mercurial-scm.org/D3487

convcmd: make a copy of heads before mutating it Differential Revision: https://phab.mercurial-scm.org/D3486

cvsps: portably convert int to bytes Differential Revision: https://phab.mercurial-scm.org/D3485

cvsps: wrap cmp methods (deprecated) in functools.cmp_to_key Differential Revision: https://phab.mercurial-scm.org/D3484

tests: port inline extension in test-convert-cvs.t to py3 # skip-blame two b prefixes, nothing more Differential Revision: https://phab.mercurial-scm.org/D3483

cvsps: add b prefixes to regular expressions # skip-blame just b prefixes Differential Revision: https://phab.mercurial-scm.org/D3482

minirst: fix bytes slicing defect on Python 3 This is the only lingering problem in minirst on Python 3. test-minirst.py now passes on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3478

tests: port test-minirst.py to Python 3 There are a few failures here, mostly around formatting lists. I'll fix that in a subsequent change. Differential Revision: https://phab.mercurial-scm.org/D3477

tests: use stringutil.pprint instead of pprint.pprint in test-minirst.py Stabilizes some output on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3476

tests: port test-pathencode.py to Python 3 # skip-blame uninteresting changes, mainly b prefixes Differential Revision: https://phab.mercurial-scm.org/D3475

tests: port test-hgwebdir-paths.py to Python 3 # skip-blame just b prefixes, nothing interesting Differential Revision: https://phab.mercurial-scm.org/D3474

tests: port test-walkrepo.py to Python 3 # skip-blame just b prefixes, nothing interesting Differential Revision: https://phab.mercurial-scm.org/D3473

tests: port test-minifileset.py to Python 3 # skip-blame all b prefixes, nothing interesting Differential Revision: https://phab.mercurial-scm.org/D3472

minifileset: fix on Python 3 Found by porting test-minifileset.py to Python 3. Differential Revision: https://phab.mercurial-scm.org/D3471

tests: port test-url.py to Python 3 # skip-blame it's all b prefixes and pycompat.bytestr() wrappers Differential Revision: https://phab.mercurial-scm.org/D3470

tests: fix error case in test-url.py's doctest This required some careful attention, so I wanted to split it out from the uninteresting bits that'll be in the next change. Differential Revision: https://phab.mercurial-scm.org/D3469

util: make util.url __repr__ consistent on Python 2 and 3 Differential Revision: https://phab.mercurial-scm.org/D3468

sslutil: fix some edge cases in Python 3 support Detected by fixing up test-url.py on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3467

tests: port test-hybridencode.py to unittest This was done predominantly through keyboard macros in emacs, so it's not always pretty, but it's enough to make the test easier to work with. There's a ton of room for improvement in this file, but it would be labor intensive and error-prone, so I went with the dumbest option that could work, so as to avoid transcription errors. Paranoia: $ egrep '^ def test' tests/test-hybridencode.py | wc -l 44 $ egrep '^ def test' tests/test-hybridencode.py | sort | uniq | wc -l so I'm pretty confident there aren't any shadowed test methods. This fixes the test on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3466

tests: make test-check-interfaces.py work on Python 3 # skip-blame just a bunch of bytes prefixes Differential Revision: https://phab.mercurial-scm.org/D3465

scmutil: clean up bytes/string cache decorator mess on Python 3 again The previous fix to this area worked, but was dropping bytes in __dict__ on Python 3. This was causing subtle breakage in test-check-interfaces.py, and probably other things too. Fixed now. Differential Revision: https://phab.mercurial-scm.org/D3464

directaccess: use resolvehexnodeidprefix() instead of _partialmatch() Same reasoning as previous commit: I want to make resolvehexnodeidprefix() move complex and don't want to duplicate that code in directaccess. Differential Revision: https://phab.mercurial-scm.org/D3463

revset: use resolvehexnodeidprefix() in id() predicate (BC) We now have a public method for this purpose, so we don't need to access the private revlog._partialmatch(). Also, I'll probably make some changes to resolvehexnodeidprefix() later, and I want those to be reflected by the id() predicate. Note that this breaks a test case, because we now resolve the prefix in the unfiltered repo and get an ambiguous lookup, which results in no revision being added to the revset. The test case was already documented to be broken even though it wasn't. It's important to note that {shortest(node)} already uses the unfiltered repo, so we're not going to break people who get the prefix from there. I think we may not want to ever use shortest() in the filtered repo. It seems unlikely to be enough of a win to matter much. For example, in my hg repo, it would save me only 0.2 hex digits. In another repo that only I modify, it saves a little more, but it's still only 0.29 hex digits. It seems unlikely that people will prune enough commits that only 1/16 of the commits are visible (which is what it would take a to save a single hex digit). Instead, I'm working on another approach: allow ambiguous matches to be disambiguated within a user-specified revset. Whether or not that pans out, I hope we're okay with this little change in behavior for now and we can decide what to do about it later. Differential Revision: https://phab.mercurial-scm.org/D3311

revset: make id() an empty set for ambiguous nodeid (BC) As Yuya pointed out in the review of D3311, id() (and rev()) does not raise an error when the input is an unknown identifier, so it doesn't make sense for it to do that when the input is ambiguous with a filtered node. However, it turned out that it already does raise an error when the input is ambiguous among the visible nodes. So let's start by fixing that. Differential Revision: https://phab.mercurial-scm.org/D3462

shortest: don't keep checking for longer prefix if node doesn't exist (API) If revlog.shortest() is called with an invalid nodeid, we keep checking if longer and longer prefixes are valid. We call revlog._partialmatch() for each prefix. That function will give us None if the node doesn't exist (and a RevlogError if it's ambiguous), so there's no need to keep checking. This patch instead makes revlog.shortest() raise a LookupError is the node does not exist, and updates the caller to handle it. Before this patch, revlog.shortest() would return the full hexnode for nonexistent nodeids. By the same reasoning as in 7b2955624777 (scmutil: make shortesthexnodeidprefix() take a full binary nodeid, 2018-04-14), it's not revlog.shortest() that should decide how to present nonexistent nodeids, so that's now moved to the template function. This should speed up cases where {shortest()} is applied to an invalid nodeid, but I couldn't think of a reasonable case where that would happen. Differential Revision: https://phab.mercurial-scm.org/D3461

shortest: extract function for checking if a prefix is a revnum Much of isvalid() was about testing if a prefix is a valid revnum. I want to reuse that soon, so let's move it out. There is no significant slowdown from the function call overhead. Differential Revision: https://phab.mercurial-scm.org/D3460

shortest: rename "test" variable to "prefix" Sorry if this is considered churn, but "prefix" just seems much clearer to me. Differential Revision: https://phab.mercurial-scm.org/D3459

shortest: move some safe code out of exception block The RevlogError and WdirUnsupported could be raised by _partialmatch(), but not by the rest of isvalid(), so let's move the rest out to make it clearer. Differential Revision: https://phab.mercurial-scm.org/D3458

revlog: don't say "not found" on internal error If index_node() returned NULL, then index_find_node() and and nt_partialmatch() used to return -2 to signal that the node was not found. However, we were passing in a revnum to index_node() that we knew should exist, so the only reason it could return NULL was due to some internal error or perhaps out of memory. Let's not use "not found" for these cases. I suppose we never noticed this because these error never happen in practice. I think there are more places where we should error out instead of reporting that the node was not found, but the cases mentioned above were all I cared about right now (because using the same error code for all failures simplified some future patches). Differential Revision: https://phab.mercurial-scm.org/D3457

revlog: extract function for getting node from known-to-exist rev Many of the calls to index_node() (which converts a rev to a nodeid) are done with a rev that's know to exist. If the function fails, there's something really wrong and we should just abort. This was done in only one place. This patch starts by extracting that code to a function that we can reuse in later patches. Differential Revision: https://phab.mercurial-scm.org/D3456

shortest: make {shortest("fffffffff")} work again {shortest("fffffffff")} should shorten it to the shortest unambiguous prefix for the working directory. It used to do that until I broke it in 7b2955624777 (scmutil: make shortesthexnodeidprefix() take a full binary nodeid, 2018-04-14), when we started returning the full hex nodeid for any working directory prefix shorter than 40 hex digits. This patch fixes it by catching WdirUnsupported specifically. Differential Revision: https://phab.mercurial-scm.org/D3455

revlog: use radix tree also for matching keys shorter than 4 hex digits I don't know what the reason for the 4-digit limit was, and I can't think of any real disadvantages of using the radix tree also when the requested minimum length is short. This speeds up `hg log -T '{shortest(node,1)}\n'` from 2m16s to 4.5s by making that not fall back to pure code. Differential Revision: https://phab.mercurial-scm.org/D3453

context: convert to hex for error message only for 20-byte changeid Now that 20-byte strings unambiguously mean binary (or a bug), we can specialize the conversion to hex for that case. Differential Revision: https://phab.mercurial-scm.org/D3452

context: clarify that only one attempt is made to interpret changeid We can now tell what type of revision specifier we have just by looking at it (we no longer attempt to interpret it in one way after the other -- that's now in scmutil.revsymbol()). Let's clarify this in the code by swithing to if/elif. Differential Revision: https://phab.mercurial-scm.org/D3451

context: only bother looking for broken dirstate for 20-byte changeid If we fail to look up a changeid in changectx.__init__, we check if it exactly matches any of the dirstate parents, and if it does, we print a more specific message ("working directory has unknown parent '...'!" instead of "unknown revision '...'"). The dirstate parents are always 20 bytes, so there's no need to check for a match when the given changeid is not 20 bytes. (And now that all the other allowed forms of changeid have been moved out of the constructor, there's no risk that a changeid that did match a dirstate parent was actually a valid bookmark.) Differential Revision: https://phab.mercurial-scm.org/D3450

context: drop support for looking up context by ambiguous changeid (API) This removes support for using the changectx constructor (and thereby repo[x]) for looking up contexts by a stringified int, a namespace key (e.g. a bookmark), or a partial hex nodeid. This means that e.g. repo[<hex nodeid>] will now fail even if a bookmark with the same name exists (which is a good thing IMO). It also means that doing repo[<non-existent node>] no longer ends up loading namespaces (which was a surprising side-effect of creating of failing to create a context object that I recently ran into while debugging something unrelated to this series). Differential Revision: https://phab.mercurial-scm.org/D3449

tests: pass parent revision as integer to repo[x] in test-context-metadata.t Support for looking up by stringified int is going away. It's already deprecated, but I think I didn't notice this case because the test case pipes through `grep`. Differential Revision: https://phab.mercurial-scm.org/D3448

bookmarks: cache reverse mapping (issue5868) I chose a simpler implementation. If the initial cost of building reverse mapping is significant, we'll have to move it under @propertycache. The nodemap could be a dict of sets, but I think keeping a sorted list is better since each node is likely to have zero/one bookmark. Micro-benchmark with 1001 bookmarks and 1001 revisions: $ for n in `seq 0 1000`; do touch $n; hg book book$n; hg ci -qAm$n; done $ hg bookmarks --time > /dev/null (orig) time: real 0.040 secs (user 0.050+0.000 sys 0.000+0.000) (new) time: real 0.040 secs (user 0.040+0.000 sys 0.010+0.000) $ hg log -T '{bookmarks}\n' --time > /dev/null (orig) time: real 0.160 secs (user 0.160+0.000 sys 0.000+0.000) (new) time: real 0.090 secs (user 0.100+0.000 sys 0.000+0.000)

bookmarks: make argument names of _set/_del() more specific

bookmarks: extract function that looks up bookmark names by node

bookmarks: hide dict behind bmstore class This should make it clearer that the bmstore doesn't expose all dict APIs.

tests: remove pid file by default Previously, killdaemons.py would kill PIDs listed in a file then leave the file lingering around. If the PIDs are killed, then there's no point leaving the PID file around. In the worst case, a later invocation of killdaemons.py (run-tests.py invokes killdaemons.py after running a test) could kill a separate process whose PID conflicted with a previously-killed process. By removing the PID file, we eliminate this possibility. Some tests were manually removing the PID file after calling killdaemons.py. So we update these tests to not do this. Differential Revision: https://phab.mercurial-scm.org/D3443

tests: remove #require killdaemons The killdaemons hghave feature has returned True since it was introduced in 448d0c452140. As such, "#require killdaemons" has no effect and is superfluous. So we remove instances of it. Differential Revision: https://phab.mercurial-scm.org/D3442

pycompat: export queue module instead of symbols in module (API) Previously, pycompat and util re-exported individual symbols from the queue module. This had the side-effect of forcing the loading of the queue module whenever pycompat/util was imported. These symbols aren't used very often. So importing the module to get a handle on the symbols is wasteful. This commit changes pycompat so it no longer exports the individual symbols in the queue module. Instead, we make the imported module a "public" symbol. We drop the individual symbol aliases from the util module. All consumers are updated to use pycompat.queue.* instead. This change makes 300 invocations of `hg log -r. -T '{rev}\n'` a little faster: before: 18.44s after: 17.87s Differential Revision: https://phab.mercurial-scm.org/D3441

demandimport: make module ignores a set (API) The list of modules to ignore is used for membership testing. Yet it is defined as a list. Sets are more efficient for membership testing. So this commit converts the module list to a set. Since we took an API hit, I renamed the variable to further clarify the change. This appears to reduce the CPU time for running 300 invocations of `hg log -r. -T '{rev}'` on my i7-6700K: before: 18.64s after: 18.44s Differential Revision: https://phab.mercurial-scm.org/D3440

cmdutil: remove unused 'confirmopts' constant Follows up f10cb49951e1.

formatter: ditch namedtuple in favor of attr

log: consume --stat/patch options at constructor of changesetprinter The variable name, self.diffopts, was confusing. Let's split it to two booleans.

log: cache diffopts instance It appears that calling patch.diff*opts() repeatedly has some cost. $ hg log -T '{rev}\n' -R mercurial --time > /dev/null (orig) time: real 4.430 secs (user 4.370+0.000 sys 0.050+0.000) (new) time: real 1.950 secs (user 1.880+0.000 sys 0.060+0.000) 'diffopts or {}' isn't necessary as patch.diff*opts() accepts opts=None.

merge with stable

encoding: alias cp65001 to utf-8 on Windows As far as I can tell, cp65001 is the Windows name for UTF-8. I don't know how different it is from the UTF-8, but Python 3 appears to have introduced new codec for cp65001, so the alias is enabled only for Python 2. https://bugs.python.org/issue13216 This patch is untested, but hopefully fixes the following issue. https://bitbucket.org/tortoisehg/thg/issues/5127/

windows: fix incorrect detection of broken pipe when writing to pager Paging e.g. hg incoming on Windows and quitting the pager before the output is consumed will print 'abort: Invalid argument'. This is because the windows error 0xE8 (ERROR_NO_DATA) is mapped to EINVAL even though it is documented as 'The pipe is being closed'. Note that this fix assumes that Windows' last error code is still valid in the exception handler. It works correctly in all my tests. A simpler fix would be to just map EINVAL to EPIPE, like was done is flush previously, but that would be less precise. This error was not observed previously, when pager was an extension.

Added signature for changeset 0b63a6743010

Added tag 4.6.2 for changeset 0b63a6743010

procutil: use unbuffered stdout on Windows Windows doesn't support line buffering, treating it as fully buffered. This causes output of slow commands to stutter. We use unbuffered instead.

merge: do not fill manifest of committed revision with pseudo node (issue5526) Since a75d24539aba "convert: fix convert dropping p2 contents during filemap merge", wctx is not always a committablectx because the convert extension passes in repo[n] as wctx. If wctx is a committed changeset, its manifest dict shouldn't be mutated reflecting to the working directory.

manifest: fix possible SEGV caused by uninitialized lazymanifest fields Before, uninitialized self->pydata would be passed to lazymanifest_dealloc() on OOM, and Py_DECREF(self->pydata) would crash if we were unlucky. It's still wrong to do malloc() thingy in tp_init because __init__() may be called more than once [1], but I don't want to go a step further in stable branch. [1]: https://docs.python.org/2/c-api/typeobj.html#c.PyTypeObject.tp_new "The tp_new function should ... do only as much further initialization as is absolutely necessary. Initialization that can safely be ignored or repeated should be placed in the tp_init handler."

tests: replace `echo -n` with `printf` per check-code Differential Revision: https://phab.mercurial-scm.org/D3749

crecord: fix line number in hunk header (issue5917) `@@ -1,1 +-1,0 @@` is not a valid patch hunk header. Change it to `@@ -1,1 +0,0 @@`. Differential Revision: https://phab.mercurial-scm.org/D3737

lazymanifest: don't crash when out of memory (issue5916) self->lines can be NULL if we failed to allocate memory for it.

cext: stop worrying and love the free(NULL) There is no need to check for a NULL pointer before calling free since free(NULL) is defined by C standards as a no-op. Lots of software relies on this behavior so it is completely safe to call even on the most obscure of systems.

tests: fix test-patch.t on pickier /bin/sh implementations This is a graft of 0b39edeff033 and f44306940c94 from default because I'm tired of seeing the FreeBSD build be red on stable. See those revisions for details on what's going on here.

chg: fix an undefined behavior about memcpy Spot by Wez Furlong. `memcpy(x, NULL, 0)` is undefined according to [1]. [1]: https://stackoverflow.com/questions/5243012 Differential Revision: https://phab.mercurial-scm.org/D3698

Added signature for changeset 9c5ced5276d6

Added tag 4.6.1 for changeset 9c5ced5276d6

mpatch: avoid integer overflow in combine() (SEC) All the callers of this function can handle a NULL return, so that appears to be the "safe" way to report an error.

mpatch: avoid integer overflow in mpatch_decode (SEC)

mpatch: fix UB integer overflows in discard() (SEC)

mpatch: fix UB in int overflows in gather() (SEC)

mpatch: introduce a safesub() helper as well Same reason as safeadd().

mpatch: introduce a safeadd() helper to work around UB int overflow We're about to make extensive use of this. This change duplicates some stdbool.h portability hacks from cext/util.h. We should probably clean that up in the future, but we'll skip that for now in order to make security backports easier.

mpatch: ensure fragment start isn't past the end of orig (SEC) Caught by oss-fuzz fuzzer during development. This defect is OVE-20180430-0004. A CVE has not been obtained as of this writing.

mpatch: protect against underflow in mpatch_apply (SEC) Also caught by oss-fuzz fuzzer during development. This defect is OVE-20180430-0002. A CVE has not been obtained as of this writing.

mpatch: be more careful about parsing binary patch data (SEC) It appears to have been possible to trivially walk off the end of an allocated region with a malformed patch. Oops. Caught when writing an mpatch fuzzer for oss-fuzz. This defect is OVE-20180430-0001. A CVE has not been obtained as of this writing.

zstandard: pull in bug fixes from upstream 0.9.1 release (issue5884) This changeset contains the meaningful code changes from python-zstandard's 0.9.1 release. The main fix is to restore support for compiling with mingw.

templatefuncs: only render text portion of minirst.format() result When "keep" argument is provided, the function returns (text, pruned), where pruned is a list of sections from the original plain text that were pruned from the rendered result. Let's not output it together with the rendered HTML.

tests: demonstrate that hgweb renders "pruned" that minirst.format() returns Notice at the bottom of the help text there's "windows". It's a section that is in the original help text, but was pruned (because hgweb didn't ask for it).