index: use `index.has_node` in `repo._rollback` Differential Revision: https://phab.mercurial-scm.org/D7329

index: use `index.has_node` in `revlog.addgroup` Differential Revision: https://phab.mercurial-scm.org/D7328

index: use `index.has_node` in `revlog.addrevision` Differential Revision: https://phab.mercurial-scm.org/D7323

index: add a `has_node` method (API) The new `index.has_node(node)` is to be preferred over: `node in index.nodemap`. This get us closer to be able to remove the `nodemap` attribute of the index. Differential Revision: https://phab.mercurial-scm.org/D7322

revlog: deal with nodemap deletion within the index Since the nodemap data now live in the index, it should be the index responsibility to ensure the data are up to date. The C version of the index is already dealing with such deletion. This work is part of a refactoring to unify the revlog index and the nodemap. This unification prepare the use of a persistent nodemap. Differential Revision: https://phab.mercurial-scm.org/D7321

revlog: clean up the node of all revision stripped in the C code For some obscure reason, the loop cleaning up node was skipping the first element… I cannot see a reason for it. The overall code is running fine nevertheless because the node are also explicitly deleted from python. We want to delete this explicit deletion, so we need to fix that code first. This work is part of a refactoring to unify the revlog index and the nodemap. This unification prepare the use of a persistent nodemap. Differential Revision: https://phab.mercurial-scm.org/D7320

revlog: move nodemap update within the index code Since the nodemap data now live in the index, it should be the index responsibility to ensure the data are up to date. This work is part of a refactoring to unify the revlog index and the nodemap. This unification prepare the use of a persistent nodemap. Differential Revision: https://phab.mercurial-scm.org/D7319

revlog: deprecate the _nodecache attribute (API) This attribute have been the same object for a long time. Code should directly access the nodemap in case of need. (Or the final API when this will be done). This work is part of a refactoring to unify the revlog index and the nodemap. This unification prepare the use of a persistent nodemap. Differential Revision: https://phab.mercurial-scm.org/D7318

revlog: access the nodemap through the index The `revlog.rev` method now access the nodemap through `self.index.nodemap`, instead of using the `_nodecache` attribute that we are trying to remove. This work is part of a refactoring to unify the revlog index and the nodemap. This unification prepare the use of a persistent nodemap. Differential Revision: https://phab.mercurial-scm.org/D7317

revlog: return the nodemap as the nodecache Ultimately, the nodecache is the nodemap. To stop having this disctinction, we start using the nodemap where the nodecache was used. This work is part of a refactoring to unify the revlog index and the nodemap. This unification prepare the use of a persistent nodemap. Differential Revision: https://phab.mercurial-scm.org/D7316

revlog: clarify which version use the older API in perf This work is part of a refactoring to unify the revlog index and the nodemap. This unification prepare the use of a persistent nodemap. Differential Revision: https://phab.mercurial-scm.org/D7315

revlog: no longer return the nodemap after parsing Now that both pure and cext version have a `nodemap` attribute, we do not need to return the `nodemap` object after parsing. This work is part of a refactoring to unify the revlog index and the nodemap. This unification prepare the use of a persistent nodemap. Differential Revision: https://phab.mercurial-scm.org/D7314

revlog: move the nodemap into the index object (for pure) This make the pure code closer to the C extension one. The ultimate goal is to merge the two into a single object and offer a unified API. This changeset focus on gathering the data on the same object. For now the code for `revlogoldindex` and `BaseIndexObject` index object are quite similar. However, there will be larger divergence later on, so I don't think is worth doing a base case. This work is part of a refactoring to unify the revlog index and the nodemap. This unification prepare the use of a persistent nodemap. Differential Revision: https://phab.mercurial-scm.org/D7313

revlog: introduce an explicit NodeMap class for pure code This class make the "pure" nodemap raise the same exception than the C-extension one. This is a step toward unifying nodemap and index, the class is not meant to survive on the long run. This work is part of a refactoring to unify the revlog index and the nodemap. This unification prepare the use of a persistent nodemap. There is a new `isinstance` call, it will be cleaned up in coming changesets. Differential Revision: https://phab.mercurial-scm.org/D7312

utils: move the `dirs` definition in pathutil (API) Before this change, the `dirs` class was accessible through the `mercurial.util` module. That module is expected to stay free of scm specific content. The `pathutil` destination has been selection by Martin von Zweigbergk. This work is part of a refactoring to unify the revlog index and the nodemap. This unification prepare the use of a persistent nodemap. Differential Revision: https://phab.mercurial-scm.org/D7311

packaging: stop installing i18n files The WiX installer has been shipping the content of the i18n/ directory since it was introduced in 2010 in 1e022c88a0a5. And the installer was subsequently refactored to only ship the .po files and hggettext. The .po files and the hggettext script are only used at build time to produce .mo files, which Mercurial does use at run-time. It doesn't make sense to install these files on Windows. So this commit stops doing that. This change further converges the file layout of the Inno and WiX installers. Differential Revision: https://phab.mercurial-scm.org/D7165

packaging: install hgk as hgk.tcl Giving it a file extension will give Windows a better chance at being able to execute it via standard means. This also makes WiX consistent with the Inno installer. Differential Revision: https://phab.mercurial-scm.org/D7164

packaging: add logo-droplets.svg The WiX installer adds this file. Let's add it to the Inno installer for consistency. Differential Revision: https://phab.mercurial-scm.org/D7163

packaging: install documentation to doc/ directory The source directory is "doc" and "doc" is used by the WiX installer. I think it makes sense to converge on the canonical path name. While we're here, we also lower the case because we can. (I think lower case names make more sense plus it is more consistent.) This only affects the Inno installer at the moment because it is the only installer using the staging code. Differential Revision: https://phab.mercurial-scm.org/D7162

packaging: always pass VERSION into Inno invocation The code in the Inno file was a holdover from before we had Python driving execution. With Python in the driver's seat, we can now have it resolve the version string and pass it into Inno, making the code easier to understand for people who aren't packaging gurus. Differential Revision: https://phab.mercurial-scm.org/D7161

packaging: remove hg.exe.local file <exe>.local files are used by Windows to set up DLL redirection. But these files are ignored if you embed an application manifest in your binary, which we do. So the existence of this file serves no purpose. So we remove it. Differential Revision: https://phab.mercurial-scm.org/D7160

packaging: stage installed files for Inno Previously, the Inno installer maintained its own mapping of source files to install location. (We have to maintain a similar mapping in the WiX installer.) Managing the explicit file layout for Windows packages is cumbersome and redundant. Every time you want to change the layout you need to change N locations. We frequently forget to do this and we only find out when people install Mercurial from our packages at release time. This commit starts the process of consolidating and simplifying the logic for managing the install layout on Windows. We introduce a list of install layout rules. These are simply source filenames (which can contain wildcards) and destination paths. The Inno packaging code has been updated to assemble all files into a staging directory that mirrors the final install layout. The list of files to add to the installer is derived by walking this staging directory and dynamically emitting the proper entries for the Inno Setup script. I diffed the file layout before and after this commit and there is no difference. Another benefit of this change is that it facilitates easier testing of the Windows install layout. Before, in order to test the final install layout, you needed to build an installer and run it. Now, you can stage files into the final layout and test from there, without running the installer. This should cut down on overhead when changing Windows code. Differential Revision: https://phab.mercurial-scm.org/D7159

packaging: process Inno Setup files with Jinja2 I want to make the Inno Setup files dynamically generated. This will enable us to do things like automatically derive the set of files to be packaged instead of having to manually keep lists of files in sync across installers. As the first step towards this, we process the Inno Setup files with Jinja2. As part of this conversion, we had to escape syntax in mercurial.iss that conflicts with Jinja2. I also took the opportunity to include modpath.iss via Jinja2 instead of using Inno's preprocessor. This keeps the Python code a bit simpler. Differential Revision: https://phab.mercurial-scm.org/D7158

packaging: install and run Inno files in a build directory Upcoming commits will refactor the Windows installers so their content is dynamically derived. In preparation for this, we copy the Inno files into a new build directory and run them from there. This required changing some relative paths to work from a new directory level but is otherwise a pretty straightforward change. Differential Revision: https://phab.mercurial-scm.org/D7157

packaging: consolidate CLI functionality into packaging.py Consolidating functionality for invoking code in the hgpackaging package through a single CLI entry point will make things simpler when we add more complexity to that package. For example, it will allow us to run things out of a virtualenv with third party packages. This commit consolidates functionality from the Inno and WiX build.py scripts into a new packaging.py script. That script simply creates a virtualenv and runs the CLI functionality in it. The new virtualenv is populated with jinja2 because I felt it easier to incorporate requirements file processing in this commit and we will soon use jinja2 in an upcoming commit. The unified CLI functionality will also make it easier to script other packaging workflows going forward. e.g. RPM, Debian, and macOS packaging. Differential Revision: https://phab.mercurial-scm.org/D7156

packaging: clean up excess packages for Inno This partially reverts e97ffa5511ba. A bunch of package dependencies were getting picked up because I ran pip-compile on Linux and the keyring package has some Linux-only dependencies. I have run pip-compile again - this time from Windows - to undo the addition of these dependencies. I also normalized whitespace to LF. Differential Revision: https://phab.mercurial-scm.org/D7155

unshelve: correct help string; unshelve does not accept file list The non-option arguments to `hg unshelve` are interpreted as the name of the shelve to unshelve, not the list of files to unshelve. While that functionality would probably be nice to have, that's well beyond the scope of this documentation fix. Differential Revision: https://phab.mercurial-scm.org/D7365

repoview: use class literal for creating filteredchangelog The type name is constant, so we don't need to create it dynamically using type(). As suggested by Yuya. Differential Revision: https://phab.mercurial-scm.org/D7364

vagrant: update vagrant image to buster from jessie I honestly have no idea if this works since I don't use vagrant, but looking around online suggests it should work fine. Differential Revision: https://phab.mercurial-scm.org/D7309

packaging: remove references to debian jessie Jessie is on life support at this point, so let's update the list of releases in the Makefiles. Differential Revision: https://phab.mercurial-scm.org/D7308

sqlitestore: remove superfluous r-prefixes on strings I did this file separately because it's a _ton_ of hits and it had some manual reformatting fallout as a result of black squeezing some expressions onto a single line. Differential Revision: https://phab.mercurial-scm.org/D7307

cleanup: remove pointless r-prefixes on single-quoted strings This is the promised second step on single-quoted strings. These had existed because our source transformer didn't turn r'' into b'', so we had tagged some strings as r-strings to get "native" strings on both Pythons. Now that the transformer is gone, we can dispense with this nonsense. Methodology: I ran hg locate 'set:added() or modified() or clean()' | egrep '.*\.py$' | xargs egrep --color=never -n -- \[\^b\]\[\^a-z\]r\'\[\^\'\\\\\]\*\'\[\^\'\ in an emacs grep-mode buffer, and then used a keyboard macro to iterate over the results and remove the r prefix as needed. # skip-blame removing unneeded r prefixes left over from Python 3 migration. Differential Revision: https://phab.mercurial-scm.org/D7306

rust-threads: force Rayon to respect the worker count in config As per the inline comment, this is a workaround because Rust code does not yet know how to read config files. Differential Revision: https://phab.mercurial-scm.org/D7310

phabricator: use context manager form of progress in uploadchunks Follow-up to 453079605242 / D7046. Differential Revision: https://phab.mercurial-scm.org/D7134

cleanup: remove pointless r-prefixes on double-quoted strings This is only double-quoted strings. I'll do single-quoted strings as a second step. These had existed because our source transformer didn't turn r"" into b"", so we had tagged some strings as r-strings to get "native" strings on both Pythons. Now that the transformer is gone, we can dispense with this nonsense. Methodology: I ran hg locate 'set:added() or modified() or clean()' | egrep '.*\.py$' | xargs egrep --color=never -n -- \[\^a-z\]r\"\[\^\"\\\\\]\*\"\[\^\"\] in an emacs grep-mode buffer, and then used a keyboard macro to iterate over the results and remove the r prefix as needed. # skip-blame removing unneeded r prefixes left over from Python 3 migration. Differential Revision: https://phab.mercurial-scm.org/D7305

formatting: remove another data-ogre from the config example The fix extension will pass the contents on stdin and read the output from stdout. By invoking `rustfmt` as `rustfmt {rootpath}`, we tell rustfmt to format the given path and update it on disk. That also makes rustftm not produce the formatted output to stdout. So the fix extension ends up writing out empty files. Differential Revision: https://phab.mercurial-scm.org/D7304

fix: replace str() by b'%d' for formatting integer Differential Revision: https://phab.mercurial-scm.org/D7303

zeroconf: improve the extension's documentation Differential Revision: https://phab.mercurial-scm.org/D7302

branchmap: pytype is confused about bytestr Differential Revision: https://phab.mercurial-scm.org/D7290

mail: suppress a pytype error that's just experimentally wrong It's probably more correct to pass a str here, but I'm not going to worry about it for now. Differential Revision: https://phab.mercurial-scm.org/D7283

dagparser: suppress some pytype errors around pycompat.bytestring I can't justify why we're getting these errors, but nothing I do fixes these handful of calls, so let's just move on with suppressions. Differential Revision: https://phab.mercurial-scm.org/D7277

encoding: add comment-based type hints for pytype Differential Revision: https://phab.mercurial-scm.org/D7275

merge with stable

dirs: reject consecutive slashes in paths We shouldn't ever see those, and the fuzzer go really excited that if it gives us a 65k string with 55k slashes in it we use a lot of RAM. This is a better fix than what I tried in D7105. It was suggested by Yuya, and I verified it does in fact cause the fuzzer to not OOM. This is a revision of D7234, but with the missing set of an error added. I added a unit test of the dirs behavior because I needed to reason more carefully about the failure modes around consecutive slashes. Differential Revision: https://phab.mercurial-scm.org/D7252

tests: quote $PYTHON in test-byteify-strings.t This is needed because the default install location is "Program Files" on Windows. Differential Revision: https://phab.mercurial-scm.org/D7251

repoview: define filteredchangelog as a top-level (non-local) class As suggested by Greg. This makes it easier for extensions to override the filtering. Differential Revision: https://phab.mercurial-scm.org/D7256

cmdutil: suppress bogus pytype errors pytype seems to handle our subclass of tuple (scmutil.status) especially poorly. We should probably file a bug. Differential Revision: https://phab.mercurial-scm.org/D7294

changegroup: suppress pytype error that's wrong Differential Revision: https://phab.mercurial-scm.org/D7293

changegroup: avoid shadowing a set with an int This confuses the heck out of pytype, and I basically agree with it that this shadowing is confusing. There's a chance this causes some memory to be freed later, but I think it's probably not worth worrying about for now. Differential Revision: https://phab.mercurial-scm.org/D7292

bundle2: more pytype suppressions around bytestring We clearly need to work around this somehow. Differential Revision: https://phab.mercurial-scm.org/D7291

archival: suppress some incorrect pytype failures Again, I suspect some missing coverage in type stubs for the stdlib. I'll revisit these later. Differential Revision: https://phab.mercurial-scm.org/D7288

simplemerge: disable a pytype error where it's just confused Differential Revision: https://phab.mercurial-scm.org/D7287

match: suppress error about subscripting an exception I get the sense I should file a bug about subscripting Exception subclasses. Differential Revision: https://phab.mercurial-scm.org/D7286

vfs: more attribute suppressions Differential Revision: https://phab.mercurial-scm.org/D7285

mdiff: mark diffopts as having dynamic attributes This class looks like we could move it to being an attrs and life would be better, but let's do that later. Differential Revision: https://phab.mercurial-scm.org/D7284

vfs: suppress some pytype errors around us using a private attribute Looking at threading._MainThread seems like we're probably a little unsupported, but since this code appears to work on both Python 2 and 3 I'm not going to sweat this for now. Differential Revision: https://phab.mercurial-scm.org/D7282

wireprototypes: disable pytype where it's just confused By inspection validnames is always a Set[bytes] here, but for some reason pytype is convinced it's a bytes. Let's disable the error for now. Differential Revision: https://phab.mercurial-scm.org/D7281

scmposix: another suppression on IOError subscripting As before, this will clear up when we move to Python 3-only. Differential Revision: https://phab.mercurial-scm.org/D7280

pvec: add an explicit type hint to help pytype Differential Revision: https://phab.mercurial-scm.org/D7279

procutil: suppress pytype warnings around windows-only attributes Differential Revision: https://phab.mercurial-scm.org/D7278