shortest: make {shortest("fffffffff")} work again {shortest("fffffffff")} should shorten it to the shortest unambiguous prefix for the working directory. It used to do that until I broke it in 7b2955624777 (scmutil: make shortesthexnodeidprefix() take a full binary nodeid, 2018-04-14), when we started returning the full hex nodeid for any working directory prefix shorter than 40 hex digits. This patch fixes it by catching WdirUnsupported specifically. Differential Revision: https://phab.mercurial-scm.org/D3455

revlog: use radix tree also for matching keys shorter than 4 hex digits I don't know what the reason for the 4-digit limit was, and I can't think of any real disadvantages of using the radix tree also when the requested minimum length is short. This speeds up `hg log -T '{shortest(node,1)}\n'` from 2m16s to 4.5s by making that not fall back to pure code. Differential Revision: https://phab.mercurial-scm.org/D3453

context: convert to hex for error message only for 20-byte changeid Now that 20-byte strings unambiguously mean binary (or a bug), we can specialize the conversion to hex for that case. Differential Revision: https://phab.mercurial-scm.org/D3452

context: clarify that only one attempt is made to interpret changeid We can now tell what type of revision specifier we have just by looking at it (we no longer attempt to interpret it in one way after the other -- that's now in scmutil.revsymbol()). Let's clarify this in the code by swithing to if/elif. Differential Revision: https://phab.mercurial-scm.org/D3451

context: only bother looking for broken dirstate for 20-byte changeid If we fail to look up a changeid in changectx.__init__, we check if it exactly matches any of the dirstate parents, and if it does, we print a more specific message ("working directory has unknown parent '...'!" instead of "unknown revision '...'"). The dirstate parents are always 20 bytes, so there's no need to check for a match when the given changeid is not 20 bytes. (And now that all the other allowed forms of changeid have been moved out of the constructor, there's no risk that a changeid that did match a dirstate parent was actually a valid bookmark.) Differential Revision: https://phab.mercurial-scm.org/D3450

context: drop support for looking up context by ambiguous changeid (API) This removes support for using the changectx constructor (and thereby repo[x]) for looking up contexts by a stringified int, a namespace key (e.g. a bookmark), or a partial hex nodeid. This means that e.g. repo[<hex nodeid>] will now fail even if a bookmark with the same name exists (which is a good thing IMO). It also means that doing repo[<non-existent node>] no longer ends up loading namespaces (which was a surprising side-effect of creating of failing to create a context object that I recently ran into while debugging something unrelated to this series). Differential Revision: https://phab.mercurial-scm.org/D3449

tests: pass parent revision as integer to repo[x] in test-context-metadata.t Support for looking up by stringified int is going away. It's already deprecated, but I think I didn't notice this case because the test case pipes through `grep`. Differential Revision: https://phab.mercurial-scm.org/D3448

bookmarks: cache reverse mapping (issue5868) I chose a simpler implementation. If the initial cost of building reverse mapping is significant, we'll have to move it under @propertycache. The nodemap could be a dict of sets, but I think keeping a sorted list is better since each node is likely to have zero/one bookmark. Micro-benchmark with 1001 bookmarks and 1001 revisions: $ for n in `seq 0 1000`; do touch $n; hg book book$n; hg ci -qAm$n; done $ hg bookmarks --time > /dev/null (orig) time: real 0.040 secs (user 0.050+0.000 sys 0.000+0.000) (new) time: real 0.040 secs (user 0.040+0.000 sys 0.010+0.000) $ hg log -T '{bookmarks}\n' --time > /dev/null (orig) time: real 0.160 secs (user 0.160+0.000 sys 0.000+0.000) (new) time: real 0.090 secs (user 0.100+0.000 sys 0.000+0.000)

bookmarks: make argument names of _set/_del() more specific

bookmarks: extract function that looks up bookmark names by node

bookmarks: hide dict behind bmstore class This should make it clearer that the bmstore doesn't expose all dict APIs.

tests: remove pid file by default Previously, killdaemons.py would kill PIDs listed in a file then leave the file lingering around. If the PIDs are killed, then there's no point leaving the PID file around. In the worst case, a later invocation of killdaemons.py (run-tests.py invokes killdaemons.py after running a test) could kill a separate process whose PID conflicted with a previously-killed process. By removing the PID file, we eliminate this possibility. Some tests were manually removing the PID file after calling killdaemons.py. So we update these tests to not do this. Differential Revision: https://phab.mercurial-scm.org/D3443

tests: remove #require killdaemons The killdaemons hghave feature has returned True since it was introduced in 448d0c452140. As such, "#require killdaemons" has no effect and is superfluous. So we remove instances of it. Differential Revision: https://phab.mercurial-scm.org/D3442

pycompat: export queue module instead of symbols in module (API) Previously, pycompat and util re-exported individual symbols from the queue module. This had the side-effect of forcing the loading of the queue module whenever pycompat/util was imported. These symbols aren't used very often. So importing the module to get a handle on the symbols is wasteful. This commit changes pycompat so it no longer exports the individual symbols in the queue module. Instead, we make the imported module a "public" symbol. We drop the individual symbol aliases from the util module. All consumers are updated to use pycompat.queue.* instead. This change makes 300 invocations of `hg log -r. -T '{rev}\n'` a little faster: before: 18.44s after: 17.87s Differential Revision: https://phab.mercurial-scm.org/D3441

demandimport: make module ignores a set (API) The list of modules to ignore is used for membership testing. Yet it is defined as a list. Sets are more efficient for membership testing. So this commit converts the module list to a set. Since we took an API hit, I renamed the variable to further clarify the change. This appears to reduce the CPU time for running 300 invocations of `hg log -r. -T '{rev}'` on my i7-6700K: before: 18.64s after: 18.44s Differential Revision: https://phab.mercurial-scm.org/D3440

cmdutil: remove unused 'confirmopts' constant Follows up f10cb49951e1.

formatter: ditch namedtuple in favor of attr

log: consume --stat/patch options at constructor of changesetprinter The variable name, self.diffopts, was confusing. Let's split it to two booleans.

log: cache diffopts instance It appears that calling patch.diff*opts() repeatedly has some cost. $ hg log -T '{rev}\n' -R mercurial --time > /dev/null (orig) time: real 4.430 secs (user 4.370+0.000 sys 0.050+0.000) (new) time: real 1.950 secs (user 1.880+0.000 sys 0.060+0.000) 'diffopts or {}' isn't necessary as patch.diff*opts() accepts opts=None.

merge with stable

encoding: alias cp65001 to utf-8 on Windows As far as I can tell, cp65001 is the Windows name for UTF-8. I don't know how different it is from the UTF-8, but Python 3 appears to have introduced new codec for cp65001, so the alias is enabled only for Python 2. https://bugs.python.org/issue13216 This patch is untested, but hopefully fixes the following issue. https://bitbucket.org/tortoisehg/thg/issues/5127/

windows: fix incorrect detection of broken pipe when writing to pager Paging e.g. hg incoming on Windows and quitting the pager before the output is consumed will print 'abort: Invalid argument'. This is because the windows error 0xE8 (ERROR_NO_DATA) is mapped to EINVAL even though it is documented as 'The pipe is being closed'. Note that this fix assumes that Windows' last error code is still valid in the exception handler. It works correctly in all my tests. A simpler fix would be to just map EINVAL to EPIPE, like was done is flush previously, but that would be less precise. This error was not observed previously, when pager was an extension.

Added signature for changeset 0b63a6743010

Added tag 4.6.2 for changeset 0b63a6743010

procutil: use unbuffered stdout on Windows Windows doesn't support line buffering, treating it as fully buffered. This causes output of slow commands to stutter. We use unbuffered instead.

merge: do not fill manifest of committed revision with pseudo node (issue5526) Since a75d24539aba "convert: fix convert dropping p2 contents during filemap merge", wctx is not always a committablectx because the convert extension passes in repo[n] as wctx. If wctx is a committed changeset, its manifest dict shouldn't be mutated reflecting to the working directory.

manifest: fix possible SEGV caused by uninitialized lazymanifest fields Before, uninitialized self->pydata would be passed to lazymanifest_dealloc() on OOM, and Py_DECREF(self->pydata) would crash if we were unlucky. It's still wrong to do malloc() thingy in tp_init because __init__() may be called more than once [1], but I don't want to go a step further in stable branch. [1]: https://docs.python.org/2/c-api/typeobj.html#c.PyTypeObject.tp_new "The tp_new function should ... do only as much further initialization as is absolutely necessary. Initialization that can safely be ignored or repeated should be placed in the tp_init handler."

tests: replace `echo -n` with `printf` per check-code Differential Revision: https://phab.mercurial-scm.org/D3749

crecord: fix line number in hunk header (issue5917) `@@ -1,1 +-1,0 @@` is not a valid patch hunk header. Change it to `@@ -1,1 +0,0 @@`. Differential Revision: https://phab.mercurial-scm.org/D3737

lazymanifest: don't crash when out of memory (issue5916) self->lines can be NULL if we failed to allocate memory for it.

cext: stop worrying and love the free(NULL) There is no need to check for a NULL pointer before calling free since free(NULL) is defined by C standards as a no-op. Lots of software relies on this behavior so it is completely safe to call even on the most obscure of systems.

tests: fix test-patch.t on pickier /bin/sh implementations This is a graft of 0b39edeff033 and f44306940c94 from default because I'm tired of seeing the FreeBSD build be red on stable. See those revisions for details on what's going on here.

chg: fix an undefined behavior about memcpy Spot by Wez Furlong. `memcpy(x, NULL, 0)` is undefined according to [1]. [1]: https://stackoverflow.com/questions/5243012 Differential Revision: https://phab.mercurial-scm.org/D3698

Added signature for changeset 9c5ced5276d6

Added tag 4.6.1 for changeset 9c5ced5276d6

mpatch: avoid integer overflow in combine() (SEC) All the callers of this function can handle a NULL return, so that appears to be the "safe" way to report an error.

mpatch: avoid integer overflow in mpatch_decode (SEC)

mpatch: fix UB integer overflows in discard() (SEC)

mpatch: fix UB in int overflows in gather() (SEC)

mpatch: introduce a safesub() helper as well Same reason as safeadd().

mpatch: introduce a safeadd() helper to work around UB int overflow We're about to make extensive use of this. This change duplicates some stdbool.h portability hacks from cext/util.h. We should probably clean that up in the future, but we'll skip that for now in order to make security backports easier.

mpatch: ensure fragment start isn't past the end of orig (SEC) Caught by oss-fuzz fuzzer during development. This defect is OVE-20180430-0004. A CVE has not been obtained as of this writing.

mpatch: protect against underflow in mpatch_apply (SEC) Also caught by oss-fuzz fuzzer during development. This defect is OVE-20180430-0002. A CVE has not been obtained as of this writing.

mpatch: be more careful about parsing binary patch data (SEC) It appears to have been possible to trivially walk off the end of an allocated region with a malformed patch. Oops. Caught when writing an mpatch fuzzer for oss-fuzz. This defect is OVE-20180430-0001. A CVE has not been obtained as of this writing.

zstandard: pull in bug fixes from upstream 0.9.1 release (issue5884) This changeset contains the meaningful code changes from python-zstandard's 0.9.1 release. The main fix is to restore support for compiling with mingw.

templatefuncs: only render text portion of minirst.format() result When "keep" argument is provided, the function returns (text, pruned), where pruned is a list of sections from the original plain text that were pruned from the rendered result. Let's not output it together with the rendered HTML.

tests: demonstrate that hgweb renders "pruned" that minirst.format() returns Notice at the bottom of the help text there's "windows". It's a section that is in the original help text, but was pruned (because hgweb didn't ask for it).

rebase: avoid RevlogError when computing obsoletenotrebased (issue5907) The key to reproducing this seems to be missing an obsolete node that is not an ancestor of the destination.

rebase: prioritize indicating an interrupted rebase over update (issue5838) This should also cover the transplant extension, and any other non clearable states.

tests: demonstrate inconsistent messaging around interrupted rebases

tests: adapt test-check-pylint to run on Windows The line endings are explicitly converted because this was ending up with '\r (no-eol) (esc)' lines, in addition to the usual '\r (esc)' lines. I've seen the fakerc output on other recently installed systems though (10.13 and/or Fedora 26). Unfortunately, the output here uses '\\' on Windows, so glob away the whole path.

hghave: avoid a deadlock reading the child process's output The output of `pylint` is voluminous enough that it fills the buffer on Windows, and waited for the parent to read it out. But the parent was waiting on the child to exit. I'm not sure what the intent of setting `ret = -1` in the exception handler just above this was...

lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902) There are only a handful of methods that access repo attributes that are applied in reposetup(). The `diff` test covers all of the commands that call scmutil.prefetchfiles(). Along the way, I saw that adding files and upgrading the repo format were also problems (also tested here). I don't think running `hg serve` through the commandserver is sane, but I conditionalized both the capabilities and the wsgirequest handler because it's trivially correct. It doesn't look like there has ever been a caller of candownload(), so there's no test for that path. The upload case isn't testable, because uploadblobs() bails if there are no pointers. The requirement should be added any time pointers are introduced, and that would force the extension to be loaded specifically for the repo. This covers `debuglfsupload`, the pre-push hook (which isn't set until the repo is promoted to LFS), and uploadblobsfromrevs(), which can be called by other extensions. I think readfromstore() and writetostore() are only reachable as a flag processor for revlog.REVIDX_EXTSTORED, and a requirement is added as soon as that is seen, so I don't think those are a problem.

help: correct signature of separate() template function Without the dots, it looked as if separate() would take a list of arguments.

hgweb: do not try to replace signal handlers while locking According to the issue 5889, mod_wsgi issues a warning on signal.signal() call, and we wouldn't want to see it in error log. The problem addressed by d77c3b023393 could potentially occur in web session, but that would be less likely than in user processes.

lock: add internal config to not replace signal handlers while locking signal.signal() is blocked in some WSGI environments, and a horrible warning is sent to the server log. So we need a way to disable it, and I think abusing ui.config is the simplest workaround.