hg-core: make parse_dirstate return references rather than update hashmaps Returing a vec is faster than updating a hashmap when the hashmap is not needed like in `hg files` which just list tracked files. Returning references avoid copying data when not needed improving performence for large repositories. Differential Revision: https://phab.mercurial-scm.org/D8861

repository: introduce constant for internal phase repo requirement and use it In future we will like to much cleaner logic around which requirement is for working copy and which can go in store. To start with that, we first need to de-clutter the requirement values spread around and replace them with constants. Differential Revision: https://phab.mercurial-scm.org/D8912

repository: introduce constant for sparse repo requirement and use it In future we will like to much cleaner logic around which requirement is for working copy and which can go in store. To start with that, we first need to de-clutter the requirement values spread around and replace them with constants. Differential Revision: https://phab.mercurial-scm.org/D8911

localrepo: refactor `.hg/requires` reading logic in separate function In an upcoming patch, we will be reusing this to read `.hg/store/requires`, so let's separate it in a different function before. Differential Revision: https://phab.mercurial-scm.org/D8910

localrepo: refactor logic to calculate sharedvfs in separate fn We will be reusing this in an upcoming patch, so better to refactor it into a separate function. Also the underlying handling deserves a function of it's own. Differential Revision: https://phab.mercurial-scm.org/D8909

store: refactor space delimited list to proper data structure There is no good reason why are having a space delimited list and then using `.split()` to get the actual list. Let's convert this into a proper collection. Differential Revision: https://phab.mercurial-scm.org/D8908

makefile: stop setting unused HGEXTDIR variable during osx build The variable was added in a38ed42cd23c (osx: include chg by default, 2017-03-20), but I can't find any others references to the variable in that commit or in any other commits. Differential Revision: https://phab.mercurial-scm.org/D8915

merge: drop commitinfo argument to applyupdates (API) We now pass the mergeresult object and hence there is no need to have a separate commitinfo argument as the required info is present in mergeresult object. Differential Revision: https://phab.mercurial-scm.org/D8904

merge: remove emptyactions() and use collections.defaultdict(list) instead emptyactions() used to return a dict which was populated and passed into applyupdates(). However, with recent changes, we no longer pass a plain dict, instead we pass the mergeresult object. There was only one usage of emptyactions and that too inside mergeresult object. That usage is replaced with collections.defaultdict(list) instead. Not sure why we were not using collections.defaultdict(list) from the beginning. Differential Revision: https://phab.mercurial-scm.org/D8903

merge: pass mergeresult obj in _forgetremoved() (API) Instead of returning a dict of actions and then updating it, let's pass the object directly and update it there. This makes `updateactions()` on mergeresult unused and this patch removes that. After this patch, we have couple of methods left on mergeresult class which still exposes the internal dict based action storage. Differential Revision: https://phab.mercurial-scm.org/D8889

mergeresult: introduce filemap() which yields filename based mapping We wanted to remove `actions` as this was leaking how we store things internally and was direct access to one of the member. This introduces filemap() which yields a map of `filename` -> `action, args, msg`. `mergeresult.actions` has been deleted as it's no longer required. Differential Revision: https://phab.mercurial-scm.org/D8888

mergeresult: add `files()` and use it `files()` will return a list of files on which an action needs to be performed. This is a step to stop exposing the underlying map to the user of this object. Differential Revision: https://phab.mercurial-scm.org/D8887

mergeresult: introduce getfile() and use it where required We want to hide the underlying dictionary from the users and provide API for valid and sane use cases. Differential Revision: https://phab.mercurial-scm.org/D8886

merge: use ACTION_* constants instead of values in _filternarrowactions() It makes easier to check what noconflicttypes are and which are not included. I hope we can have a state where we always use ACTION_* constants instead of these values which are very hard to understand. Differential Revision: https://phab.mercurial-scm.org/D8885

merge: rework iteration over mergeresult object in checkpathconflicts() Instead of following pattern: ``` for f, (m, args, msg) in mresult.actions.items(): if m == mergestatemod.ACTION_*: ... elif m == mergestatemod.ACTION_*: ... .... ``` We do: ``` for (f, args, msg) in mresult.getaction((mergestatemod.ACTION_*,)): ... for (f, args, msg) in mresult.getaction((mergestatemod.ACTION_*,)): ... .... ``` This makes code bit easier to understand and prevent iterating over actions which we don't need. Differential Revision: https://phab.mercurial-scm.org/D8884

applyupdates: simplfy calculation of number of updated files Instead of increasing the `updated` variable each time in a loop, let's use the mergeresult object API to calculate the updated value in one call. Differential Revision: https://phab.mercurial-scm.org/D8883

mergeresult: yield from getactions() instead of buidling a list and returning Only 7 out of 29 callers change the underlying dict while iterating. So it's better to yield and wrap the 7 callers with `list()`. Differential Revision: https://phab.mercurial-scm.org/D8882

sparse: replace merge action values with mergestate.ACTION_* constants Having bytestrings like `b'r'` makes it hard to understand for people who don't know the code much or looking at it for the first time. Differential Revision: https://phab.mercurial-scm.org/D8881

mergeresult: implement a len() function and use it In next patch we will start yielding from `getactions()` instead of building and returning a list. Hence we can no longer rely on that for getting us a length. Differential Revision: https://phab.mercurial-scm.org/D8880

merge: replace use of actions dict with mergeresult object There are still some places which can be improved by having a dedicated API, this patch for now make all users of actions dict go through the mergeresult object API. Differential Revision: https://phab.mercurial-scm.org/D8879

mergeresult: add sort argument to getactions() method This will be used in next patch. Differential Revision: https://phab.mercurial-scm.org/D8878

merge: pass mergeresult obj in merge._prefetchfiles() Differential Revision: https://phab.mercurial-scm.org/D8877

merge: pass mergeresult obj instead of actions in applyupdates() (API) This is similar to past 20 patches or so where we are replacing use of a bare actions dict with a dedicated mergeresult object. The goal is to have a dedicated powerful object instead of a dict while making the code more easier to understand. In past few patches, we have already simplified the code at some places using the newly introduced object. This patch does not updates applyupdates() to use the mergeresult object directly. That will be done in next patch to make things easier to review. Differential Revision: https://phab.mercurial-scm.org/D8876

merge: pass mergeresult obj instead of actions in _checkcollision() (API) The goal is to not use the actions dict and replace it with a rich mergeresult object. Differential Revision: https://phab.mercurial-scm.org/D8875

commitctx: directly update the touched and added set of files Instead of going through intermediate variable, we can simply use the ChangingFiles object. The object will take care of the consistency.

commitctx: directly updated the set of removed files The change is non-trivial so I made it in its own changeset.

commitctx: create the `ChangingFiles` object sooner Let us change the `_commit_manifest` call before other changes.

commitctx: move ChangingFiles creation directly inside `_process_files` As announced, we move it there. We focus on the signature change first, we will update the code afterward.

commitctx: create the ChangingFiles object directly in the various case No need to compute all data then create the object, we can create it early and directly store data in it. We start simple by moving create higher in the function, but the end goal is to eventually move the creation inside the `_process_files` function to take advantage of the object there.

commitctx: no longer make the storage and added/removed file optional The code using this variable is always using other, stricter, condition before using these value. So it is safe to always carry them along.

commitctx: directly gather p1 and p2 copies in `files` The only thing we do with the p1copies and p2copies is to pass them around, we we can gather them later and directly stored them in the `ChangingFiles` object.

commitctx: extract sidedata encoding inside its own function This part of the code is quite independent from the rest. Thank to the new ChangingFiles object, moving with the rest of the sidedata code (in metadata.py) is simple. The changelog.add method is simply passing the `files` object to the new function. It will be easy to increase/change the data we gather and encode without impacting the changelog method.

commitctx: directly pass a ChangingFiles object to changelog.add We pass the rich object to the changelog and it read the field it needs.

commitctx: return a richer object from _prepare_files Instead of returning a lot of different list, we introduce a rich object that hold all the file related information. The unique object help with data consistency and simply functions arguments and return. In the rest of this series we will increase usage of this object to simplify more code.

templater: teach template loader to use open_template() function The template loader can apparently load templates from relative paths, so I needed to add that support to `open_template()`. This takes the number of failing tests with PyOxidizer from 54 to 34. Differential Revision: https://phab.mercurial-scm.org/D8907

templater: restructure open_template() a little to prepare for relative paths I found that it was easier to add support for relative paths after this restructuring. It also made it easier to explain each case with a code comment (which I did). Differential Revision: https://phab.mercurial-scm.org/D8906

templater: add exception-raising version of open_template() I'm about to add another caller of `open_template()` (in the template loader). That caller will want to get exceptions instead of `(None, None)` if the template doesn't exist. This patch therefore changes `open_template()` to raise exceptions and adds a new `try_open_template()` that returns the `(None, None)` value. Differential Revision: https://phab.mercurial-scm.org/D8905

templater: replace Py3-only exception types by super-types available in Py2 As noted by @indygreg, `test-check-pyflakes.t` started failing on Py2 after my recent D8894, because that introduced catching of the Py3-only types `ModuleNotFoundError` and `FileNotFoundError`. Let's switch to less precise types that are also available in Py2. Differential Revision: https://phab.mercurial-scm.org/D8902

hgweb: enable reading styles from resources in frozen binaries All we need to do to read styles from resources is to pass the file-like object we get from `open_template()` on to `frommapfile()`. This takes the number of failing tests with PyOxidizer from 62 to 54. Differential Revision: https://phab.mercurial-scm.org/D8901

hgweb: rely on open_template()'s fallback to using templatedir() Differential Revision: https://phab.mercurial-scm.org/D8900

hgweb: open mapfile using templater.open_template() This will help us read templates from resources in frozen binaries. Differential Revision: https://phab.mercurial-scm.org/D8899

hgweb: simplify a constant-length list by converting to literal tuple The call to `.append()` has been unnecessary since d3dbdca92458 (hgweb: don't choke when an inexistent style is requested (issue1901), 2009-11-12). Differential Revision: https://phab.mercurial-scm.org/D8898

hgweb: remove some accesses to private member uimod._unset Differential Revision: https://phab.mercurial-scm.org/D8897

templater: try to read %include in mapfiles from resources The "show" style is an example of a style that uses an "%include" statement in its definition. This patch makes `hg log --style show` work. This takes the number of failing tests with PyOxidizer from 72 to 62. Differential Revision: https://phab.mercurial-scm.org/D8896

templater: unroll loop over mapfile directories I'll rewrite the handling of the `templatedir()` case in the next patch, so the two cases will be more different and the loop won't make as much sense. Differential Revision: https://phab.mercurial-scm.org/D8895

templater: make open_template() read from resources if in frozen binary This takes the number of failing tests with PyOxidizer from 87 to 72. Differential Revision: https://phab.mercurial-scm.org/D8894

templater: pass opened file-like object to templatespec I think I said earlier that I planned to create a special templatespec variant for built-in templates. That was true (I planned that), but I ended up (in this patch) just adding a file-like object to the `mapfile_templatespec()` variant instead. Differential Revision: https://phab.mercurial-scm.org/D8893

templater: replace templatepath() with function that also opens the file For frozen binaries, such as those created by PyOxidizer, I plan to make it so the templatespec can keep an opened file/resource to read from instead of needing a file path. Having `templatepath()` return an opened file should help with that. At this point, it's just a wasteful extra opening of mapfiles that we'll open again later. I'll update the read-side next so it reads from the file-like object without opening the file again. Differential Revision: https://phab.mercurial-scm.org/D8892

templater: start passing resource to read from into _readmapfile() This patch makes it so we pass in a file-like resource to read from instead of having `_readmapfile()` open the file. This is one more step towards making `_readmapfile()` able to read resources opened by from `importlib.resources`. We still need to pass in the mapfile path because it's used for loading dependent mapfiles from `%include` and `__base__`, and it's also used for giving the user better error messages. Besides that, one can safely call `_readmapfile()` with any file-like resource after this patch. Differential Revision: https://phab.mercurial-scm.org/D8891

templater: move stylemap() to hgweb_mod, since that's its only user `stylemap()` even has an error message that mentions "hgweb templates", so it seems that it's meant specifically for hgweb. Differential Revision: https://phab.mercurial-scm.org/D8890

hgweb: simplify staticfile() now that we always pass it a single directory I didn't realize this further simplifications enabled by D8786 until now. Differential Revision: https://phab.mercurial-scm.org/D8874

packaging: include templates with their package as key in package_data This is similar to an earlier patch in this series. It seems more correct to use `mercurial.templates.coal` etc as keys in the `package_data` dict now that those modules are packages. Differential Revision: https://phab.mercurial-scm.org/D8858

packaging: mark mercurial.templates and subdirs as packages We need these packages to be installed so PyOxidizer picks them up. Differential Revision: https://phab.mercurial-scm.org/D8855

templates: add __init__.py files to templates/ dirs This is necessary for them to be loaded with `importlib.resources`, which we want to do for PyOxidizer and similar. `importlib.resources` cannot read resources from submodules (`resources.open_binary('mercurial.templates', 'coal/map')` is not valid), so we need one `__init__.py` per directory. Differential Revision: https://phab.mercurial-scm.org/D8854

merge with stable

localrepo: abort creating a shared repo if the source does not have store We cannot create a shared repository without a store IIUC. Let's abort in such cases. Differential Revision: https://phab.mercurial-scm.org/D8772

localrepo: only use 'bookmarksinstore' requirement if we have 'store' This adds check that whether we have the 'store' requirement or not. If we don't have that, we skip adding the 'bookmarksinstore' requirement and warn user about it. Differential Revision: https://phab.mercurial-scm.org/D8771

mergeresult: make actionmapping a dict of dict instead of dict of lists This makes deleting a specific filename entry faster and easier. Differential Revision: https://phab.mercurial-scm.org/D8837

largefiles: introduce a constant for 'lfmr' action It's better to use a dedicated constant instead of a string which makes pretty less sense. Differential Revision: https://phab.mercurial-scm.org/D8836

largefiles: override merge.emptyactions() to include `lfmr` I found it weird that we were not already doing this. I encountered this while using `emptyactions()` in mergeresult() class. Differential Revision: https://phab.mercurial-scm.org/D8835