diff: disable diff.noprefix option for diffstat (issue5759) We could use patch.diffhunks() instead of patch.diff() to get filenames without parsing patch content, but that isn't always possible because we sometimes feed raw patch data to patch.diffstat().

hgweb: disable diff.noprefix option for diffstat Copied from cf1e15f91c90.

workers: handling exceptions in windows workers This adds handling of exceptions from worker threads and resurfaces them as if the function ran without workers. If any of the threads throws, the main thread kills all running threads giving them 5 sec to handle the interruption and raises the first exception received. We don't have to join threads if is_alive() is false Test Plan: Ran multiple updates/enable/disable sparse profile and things worked well Ran test on CentOS- all tests passing on @ passed here Added a forged exception into the worker code and got it properly resurfaced and the rest of workers killed: P58642088 PS C:\open\<repo>> ..\facebook-hg-rpms\build\hg\hg.exe --config extensions.fsmonitor=! sparse --enable-profile <profile> updating [==> ] 1300/39166 1m57sException in thread Thread-3: Traceback (most recent call last): File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\threading.py", line 801, in __bootstrap_inner self.run() File "C:\open\facebook-hg-rpms\build\hg\mercurial\worker.py", line 244, in run raise e Exception: Forged exception Exception in thread Thread-2: Traceback (most recent call last): File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\threading.py", line 801, in __bootstrap_inner self.run() File "C:\open\facebook-hg-rpms\build\hg\mercurial\worker.py", line 244, in run raise e Exception: Forged exception <...> Traceback (most recent call last): File "C:\open\facebook-hg-rpms\build\hg\hgexe.py", line 41, in <module> dispatch.run() File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 85, in run status = (dispatch(req) or 0) & 255 File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 173, in dispatch ret = _runcatch(req) File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 324, in _runcatch return _callcatch(ui, _runcatchfunc) File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 332, in _callcatch return scmutil.callcatch(ui, func) File "C:\open\facebook-hg-rpms\build\hg\mercurial\scmutil.py", line 154, in callcatch return func() File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 314, in _runcatchfunc return _dispatch(req) File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 951, in _dispatch cmdpats, cmdoptions) File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\remotefilelog\__init__.py", line 415, in runcommand return orig(lui, repo, *args, **kwargs) File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\hgext3rd\undo.py", line 118, in _runcommandwrapper result = orig(lui, repo, cmd, fullargs, *args) File "C:\open\facebook-hg-rpms\build\hg\hgext\journal.py", line 84, in runcommand return orig(lui, repo, cmd, fullargs, *args) File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\hgext3rd\perftweaks.py", line 268, in _tracksparseprofiles res = runcommand(lui, repo, *args) File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\hgext3rd\perftweaks.py", line 256, in _trackdirstatesizes res = runcommand(lui, repo, *args) File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\hgext3rd\copytrace.py", line 144, in _runcommand return orig(lui, repo, cmd, fullargs, ui, *args, **kwargs) File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\hgext3rd\fbamend\hiddenoverride.py", line 119, in runcommand result = orig(lui, repo, cmd, fullargs, *args) File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 712, in runcommand ret = _runcommand(ui, options, cmd, d) File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 959, in _runcommand return cmdfunc() File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 948, in <lambda> d = lambda: util.checksignature(func)(ui, *args, **strcmdopt) File "C:\open\facebook-hg-rpms\build\hg\mercurial\util.py", line 1183, in check return func(*args, **kwargs) File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\hgext3rd\fbsparse.py", line 860, in sparse disableprofile=disableprofile, force=force) File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\hgext3rd\fbsparse.py", line 949, in _config len, _refresh(ui, repo, oldstatus, oldsparsematch, force)) File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\hgext3rd\fbsparse.py", line 1116, in _refresh mergemod.applyupdates(repo, typeactions, repo[None], repo['.'], False) File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\remotefilelog\__init__.py", line 311, in applyupdates return orig(repo, actions, wctx, mctx, overwrite, labels=labels) File "C:\open\facebook-hg-rpms\build\hg\mercurial\merge.py", line 1464, in applyupdates for i, item in prog: File "C:\open\facebook-hg-rpms\build\hg\mercurial\worker.py", line 286, in _windowsworker raise t.exception Exception: Forged exception PS C:\open\ovrsource> Differential Revision: https://phab.mercurial-scm.org/D1459

workers: implemented worker on windows This change implements thread based worker on windows. The handling of exception from within threads will happen in separate diff. The worker is for now used in mercurial/merge.py and in lfs extension After multiple tests and milions of files materiealized, thousands lfs fetched it seems that neither merge.py nor lfs/blobstore.py is thread unsafe. I also looked through the code and besides the backgroundfilecloser (handled in base of this) things look good. The performance boost of this on windows is ~50% for sparse --enable-profile * Speedup of hg up/rebase - not exactly measured Test Plan: Ran 10s of hg sparse --enable-profile and --disable-profile operations on large profiles and verified that workers are running. Used sysinternals suite to see that all threads are spawned and run as they should Run various other operations on the repo including update and rebase Ran tests on CentOS and all tests that pass on @ pass here Differential Revision: https://phab.mercurial-scm.org/D1458

workers: don't use backgroundfilecloser in threads This disables background file closing when in not in main thread Test Plan: Ran pull, update, sparse commands and watched the closer threads created and destroyed in procexp.exe ran test on CentOS. No tests broken compared to the base Differential Revision: https://phab.mercurial-scm.org/D1457

debugssl: convert port number to int (issue5757) It doesn't use util.getport(), which may resolve service name to port number.

debugdiscovery: correct and clean up command synopsis This does a few things: * Changes "-r" to "--rev", since "-r" is not a valid short form * Removes non-existent "-l" and "-b" options * Removes "..." after options, since we don't usually have that Differential Revision: https://phab.mercurial-scm.org/D1706

copies: group wdir-handling in one place I think this makes it both easier to follow and shorter. Differential Revision: https://phab.mercurial-scm.org/D1698

copies: extract method for getting non-wdir forward copies I may add an alternative way of getting copy metadata (from changelog, not filelog) but the chaining with the dirstate copy metadata will be the same, so it will probably help to have this extracted. Even if that doesn't happen, the next patch will show that we can simplify this a bit after this refactoring, so it seems worth it regardless. Differential Revision: https://phab.mercurial-scm.org/D1697

copies: consistently use """ for docstrings Differential Revision: https://phab.mercurial-scm.org/D1696

copies: always respect matcher arg to _forwardcopies() The function would ignore the matcher if the dirstate copies were requested. It doesn't matter in practice because all callers used the returned map only for looking up specific files from and those files had already been filtered by the matcher (AFACT). Still, it's a little confusing, so let's make it clearer by respecting the matcher in this case too. Differential Revision: https://phab.mercurial-scm.org/D1695

debugdiscovery: drop reference to non-existent --remote-head option It seems like it didn't even exist when debugdiscovery was introduced in cb98fed52495 (discovery: add new set-based discovery, 2011-05-02). Differential Revision: https://phab.mercurial-scm.org/D1693

debugdiscovery: drop reference to non-existent --serverlog option It seems like it didn't even exist when debugdiscovery was introduced in cb98fed52495 (discovery: add new set-based discovery, 2011-05-02). Differential Revision: https://phab.mercurial-scm.org/D1692

debugdiscovery: drop reference to invalid --branch option It seems like it didn't even exist when debugdiscovery was introduced in cb98fed52495 (discovery: add new set-based discovery, 2011-05-02). Differential Revision: https://phab.mercurial-scm.org/D1691

tests: avoid echo with backslash escapes Some shells' echo (e.g. Bash's) do not interpret backslash escapes by default, while others shells' do (e.g. Dash's). This led to a failure in test-run-tests.t. We could switch to the more compatible printf, but using a heredoc seems simpler, so that's what this patch does. For anyone who wants to test this for themselves, remove the "unset HGTEST_SHELL" line in helper-runtests.sh and pass e.g. "--shell=/bin/dash" to the test runner. Differential Revision: https://phab.mercurial-scm.org/D1699

hgweb: stop using HTML comments in <script> Once upon a time, in 1995, there were browsers that didn't understand <script> tags and they would simply show the code inside as text. This started a tradition of wrapping everything inside <script> in <!-- HTML comments -->. Nowadays, it's not only not needed, but can be considered harmful[1]: - within XHTML documents, the source will actually be hidden from all browsers and rendered useless - `--` is not allowed within HTML comments, so any decrement operations in script are invalid [1]: http://www.javascripttoolbox.com/bestpractices/#comments

merge with stable

statprof: remove a line of deadcode Differential Revision: https://phab.mercurial-scm.org/D1689

extdata: abort if external command exits with non-zero status (BC) Per the last discussion, this is more reliable and consistent way than suppressing an error. For grep, erroring out might be inconvenient, but for curl, non-zero exit status should be detected. The latter wouldn't be possible if non-zero status is ignored. https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-October/105727.html

hgweb: update graph function docstring

rebase: fix for hgsubversion 5c25fe7fb1e broke something in the hgsubversion test path, causing it raise an abort (Abort: nothing to merge) during a perfectly good rebase. I tracked it down to this change. It's probably not hgsubversion related. I suspect that using the same `wctx` from before the initial update causes problems with the wctx's cached manifest property. I noticed we also sometimes stick random gunk on the wctx object in other places (like in `copies.py`) so it's probably best to reset it for now. The line I added before was actually useless since we don't pass wctx to the initial `merge.update`, so it defaults to `repo[None]`. So I just removed it. Differential Revision: https://phab.mercurial-scm.org/D1679

hgweb: render next pages on /graph incrementally Previously, when user scrolled down to see the next page on /graph, all hgweb did was re-render everything that would be visible (by simply incrementing revcount). It was not efficient at all, and this patch makes /graph page behave similarly to the regular /log: every new page only consists of new changesets, no duplication, and only jsdata is based on the full set of changesets required to build accurate graph. This is achieved by adding "?graphtop=<node>" to the next page URL template, effectively remembering where the graph started, and using that value to create the new `tree` that covers the whole visible graph. That variable is then used to produce jsdata for redrawing graph client-side. nextentry is used for the same purpose as on /log page (to format the next page URL), but it's not a part of the graph.

hgweb: split graphdata() into jsdata() and nodes() nodes keyword passed to the template can be any iterator, but jsdata needs to be a list because it gets JSONified.

hgweb: calculate <canvas> width and height client-side hgweb determines and passes to templates some variables related to graph appearance, like bg_height, canvaswidth and canvasheight. bg_height was and still is used for graph.scale() call in graph.tmpl, and the two latter variables were used in <canvas> element as width and height properties, and they were set before JS code got to run. Setting these properties server-side doesn't make a lot of sense, because a graph that has been scaled should calculate things like width and height on its own when being rendered. Let's move (re)sizing <canvas> to JavaScript (to Graph.render function) and stop parsing HTML with regular expressions just to know new width and height. That extra loop that only counts cols is required because <canvas> can't be resized after or in the process of rendering (or it gets cleared). Incidentally, SVG doesn't have this problem and I'm hoping to switch graph to using it in future. There also was truecanvasheight, but according to hg grep --all it was never used, see d490edc71146.