phase: report number of non-public changeset alongside the new range When interacting with non-publishing repository or bundle, it is useful to have some information about the phase of the changeset we just pulled. This changeset updates the "new changesets MIN:MAX" output to also includes phases information for non-public changesets. Displaying extra data about non-public changesets means the output for exchange with publishing repository (the default) is unaffected.

tests: disable test-nointerrupt on Windows Per the followup discussion[1]. proc.send_signal(INT) in timeout.py raises a ValueError because of an unsupported signal. I don't like missing test coverage for this on Windows. But this is the last test failing on Windows, and red all the time hides new failures. [1] https://phab.mercurial-scm.org/D3716

tests: conditionalize an error message about unlinking a non empty directory The message on Windows comes from win32.unlink(). It looks like os.unlink() on posix platforms is a simple call to unlink(3), which turns into unlinkat(2). Since there's a comment in one of the tests that the message should be improved, I don't think it's worth adding a check in win32.unlink() to see if it's empty, if that function is always going to fail on a directory. (It seems like the POSIX spec allows unlinking directories though.)

ancestors: add nullrev to set from the beginning Differential Revision: https://phab.mercurial-scm.org/D4507

ancestor: filter out initial revisions lower than stoprev

ancestor: add test showing inconsistency between __iter__ and __contains__

ancestors: ensure a consistent order even in the "inclusive" case It seems odds to first issue the "source" revs and then the other ancestors. In addition, doing so can break the other contract of always issuing a child before its parent. We update the code to apply the same logic to all yielded revision. No tests break so we seem in the clear except where we explicitly test the order.

ancestors: actually iterate over ancestors in topological order (issue5979) This code previously used a dequeue logic, the first ancestors seen were the first ancestors to be emitted. In branching/merging situations, it can result in ancestors being yielded before their descendants, breaking the object contract. We got affected by this issue while working on the copy tracing code. At about the same time, Axel Hecht <axel@mozilla.com> reported the issue and provided the test case used in this changeset. Thanks Axel. Running `hg perfancestors` does not show a significant difference between the old and the new version.

doc: use modern import style in runrst

hgweb: do not audit URL path as working-directory path Since hgweb is an interface to repository data, we don't need to prohibit any paths conflicting within the filesystem. Still an access to working files is audited by filectx.

hgweb: map Abort to 403 error to report inaccessible path for example Abort is so common in our codebase. We could instead introduce a dedicated type for path auditing errors, but we'll probably have to catch error.Abort anyway. As you can see, an abort message may include a full path to the repository, which might be considered information leak. If that matters, we should hide the message and send it to the server log instead.

hgweb: add error template to json so it won't crash

hgweb: show shortlog by default in json output (issue5978)

merge with stable

tests: improve the widening testing in test-narrow-widen* Before this patch, we are testing `hg tracked --addinclude` by adding a command which is not introduced in the changesets till now. If you closely look at the tests, wider/f was introduced on the server after the narrow clone was done and extending the existing clone to include wider/f does not make sense. We should test extending a file which exists. Differential Revision: https://phab.mercurial-scm.org/D4452

narrow: use util.readfile() and improve error message using --narrowspec This patch improves the error message and uses util.readfile() for reading narrowspecs file while cloning. Differential Revision: https://phab.mercurial-scm.org/D4462

merge: use vfs methods for I/O All I/O is supposed to be performed via vfs instances so filesystems can be abstracted. The previous commit ported the old code in purge, which didn't go through the vfs layer. This commit ports the purge code to use the vfs layer. The vfs layer didn't have a method to remove a single directory, so it was added as part of implementing this. Differential Revision: https://phab.mercurial-scm.org/D4478

merge: move purge logic from extension Working directory purging feels like functionality that should be in core rather than in an extension. This commit effectively moves the core purging logic from the purge extension to merge.py. Code was refactored slightly. Rather than deal with printing in this function, the function is a generator of paths and the caller can worry about printing. Differential Revision: https://phab.mercurial-scm.org/D4477

tests: stabilize test-removeemptydirs.t on Windows

help: add internals.wireprotocolv2 to the table, and remove redundant header

match: improve includematcher.visitchildrenset to be much faster and cached This improves the speed of visitchildrenset considerably, especially when there are complicated matchers involved that may have many entries in _dirs or _parents. Unfortunately the benchmark isn't easily upstreamed due to its reliance on https://github.com/vstinner/perf (primarily due to the conflict when importing it if I were to contribute the benchmark as contrib/matcherbenchmarks.py) instead of asv or some other perf measurement system. To describe the benchmark briefly: I generated an includematcher of either 5 or 3500 "rootfilesin:prefix1/prefix2/prefix3/<randomsubdirs, 1-8 levels deep>" items in the 'setup' function, and then called `im.visitchildrenset('prefix1/prefix2')` in the 'stmt' function in perf.timeit. For the set of 5: - before: 15.3 us +- 2.9 us - after: 1.59 us +- 0.02 us For the set of 3500: - before: 3.90 ms +- 0.10 ms - after: 3.15 us +- 0.09 us (note the m->u change) Differential Revision: https://phab.mercurial-scm.org/D4351

py3: add new passing tests spotted by the buildbot Differential Revision: https://phab.mercurial-scm.org/D4495

tests: order the imports in test-fastannotate-hg.t The wrong ordering breaks test-check-module-imports.t on Python 3. I am not sure why that test is so much active on py3. Differential Revision: https://phab.mercurial-scm.org/D4496

lfs: ensure the blob is linked to the remote store on skipped uploads I noticed a "missing" blob when pushing two repositories with common blobs to a fresh server, and then running `hg verify` as a user different from the one running the web server. When pushing the second repo, several of the blobs already existed in the user cache, so the server indicated to the client that it doesn't need to upload the blobs. That's good enough for the web server process to serve up in the future. But a different user has a different cache by default, so verify complains that `lfs.url` needs to be set, because it wants to fetch the missing blobs. Aside from that corner case, it's better to keep all of the blobs in the repo whenever possible. Especially since the largefiles wiki says the user cache can be deleted at any time to reclaim disk space- users switching over may have the same expectations.

cbor: teach the encoder to handle python `long` type for Windows The tests for 2**32 and -7000000000 were blowing up, complaining about not knowing how to encode type 'long'. sys.maxint tops out at 2**31-1 on Windows, but I guess is 2^63-1 on Linux? I *think* we're OK on the decode side, as there is an assertion that the decoded value is equal to the original primitive value. I opted for the pycompat alias instead of swallowing the NameError because the vendored cbor package uses an alias, and I see at least pywatchman and templatefilters open codes their own aliases.

buildrpm: remove TODO that looks done to me Differential Revision: https://phab.mercurial-scm.org/D4494

buildrpm: fix embarassing bug in shell variable expansions `$distance_$node` looks up and concatenates `$distance_` and `$node`. `$distance_` is empty, so we were getting the node without the distance. Using the curly braces makes our intent explicit and produces better-versioned RPMS. Differential Revision: https://phab.mercurial-scm.org/D4493

absorb: clarify the reason for not finding changesets to modify I'm used to pushing to non-publishing repos, so this was the last thing I thought to check.

merge with stable Includes an extra bump of the version number for parsers because the merge produces a new, distinct version of the code.

state: use our CBOR module This was the last consumer of the vendored CBOR package in core. Differential Revision: https://phab.mercurial-scm.org/D4471

wireprotov2peer: use our CBOR decoder Behavior should be the same in order to preserve backwards compatibility. We obviously want to stream values in this code. We'll do that in subsequent commits. Differential Revision: https://phab.mercurial-scm.org/D4470

debugcommands: use our CBOR decoder It implements the set of CBOR needed for the wire protocol. Differential Revision: https://phab.mercurial-scm.org/D4469

wireprotov2server: use our CBOR encoder Again, test output changed slightly because of map key ordering differences. This shouldn't matter. I could have called oncommandresponsereadygen() with the raw output from the stream encoder. However, this changed test output further. I left a TODO to follow up on that later. Differential Revision: https://phab.mercurial-scm.org/D4468

wireprotoserver: use our CBOR encoder Moving away from the vendored package. Again, our encoder normalizes keys differently from the vendored package (for now), hence the test changes. Differential Revision: https://phab.mercurial-scm.org/D4467

wireprotoframing: use our CBOR module Tests changed because our CBOR encoder appears to sort map keys differently from the vendored CBOR package. The CBOR specification does define canonical sorting rules for keys based on the byte values. I'm guessing our implementation doesn't follow them. But our encoder doesn't guarantee that it conforms with the canonical specification. Right now, we just care that output is deterministic. And our encoder does guarantee that. Differential Revision: https://phab.mercurial-scm.org/D4466

httppeer: use our CBOR decoder We just implemented our own CBOR decoder. Let's use it in httppeer. Differential Revision: https://phab.mercurial-scm.org/D4465

zsh_completion: complete shelve and unshelve Differential Revision: https://phab.mercurial-scm.org/D4428

zsh_completion: use revsets to exclude this rev from suggestions to hg merge One of the most important aspects of a completion system is its speed, so 1 call to hg is definitely better than 4. Sorting by rev (descending) is to preserve the same order as in `hg heads` output. While at it, declare branches as an array too. Differential Revision: https://phab.mercurial-scm.org/D4426

drawdag: correctly pass repo to super constructor This was caught by someone who was looking for the super(cls, self).__init__(self, ...) pattern in all code in our Google-internal repo. Differential Revision: https://phab.mercurial-scm.org/D4464

help: correct a typo in tweakdefaults

narrow: add '--import-rules' flag to tracked command This patch adds a `--import-rules` flag to tracked command provided by narrow extension. Using the --import-rules flag, you can pass a filename from which narrowspecs should be read and added to main narrowspec. A lot of times, in automation or manually also, when you are working with big repo, specifying each path name on commandline using '--addinclude' and '--addexclude' is tedious and something which can scale. So we needed something where we can pass a file to extend the narrowspecs. Nice thing about this is that the automations which reads some file to change the sparse profile, can now read the same file for changing narrowspecs too. Tests are added for the new feature. Differential Revision: https://phab.mercurial-scm.org/D4125

internals: extract wire protocol version 2 commands to standalone doc wireprotocol.txt is a bit long and unwieldy. I think splitting it up will help aid comprehension. Let's start by extracting wire protocol version 2 commands to a standalone document. As part of the port, I munged with the section titles a bit and expanded the TODO around node namespaces. Differential Revision: https://phab.mercurial-scm.org/D4442

wireprotov2peer: split responsedata handling into separate function So we don't have so much logic inside an if/elif block. Differential Revision: https://phab.mercurial-scm.org/D4439

httppeer: add TODO about delayed handling of ^C When testing wire protocol v2 with large responses, I noticed that ^C doesn't result in immediate termination. I'm not sure what the root cause is. Let's add a TODO so we don't forget about it. Differential Revision: https://phab.mercurial-scm.org/D4438

httppeer: log commands for version 2 peer This will aid testing and debugging. Differential Revision: https://phab.mercurial-scm.org/D4437

wireprotov2peer: add TODO about streaming responses This is a pretty big deficiency in the current implementation. We don't want to forget about this. Differential Revision: https://phab.mercurial-scm.org/D4435

tests: update relnotes test for actual output I'm unclear why I didn't see this before but I consistently see it now. In any event, it doesn't appear to be a regression, so let's just bake it into the test. Differential Revision: https://phab.mercurial-scm.org/D4463

py3: use print as a function in tests/test-hgweb.t Differential Revision: https://phab.mercurial-scm.org/D4461