histedit: modernize write of histedit-last-edit file Differential Revision: https://phab.mercurial-scm.org/D2241

histedit: convert bool to bytestring manually Yes, this format has 'True' and 'False' in it. I'm sorry, as it's almost certainly my fault. Differential Revision: https://phab.mercurial-scm.org/D2240

narrowcommands: more byteskwargs cleanup Differential Revision: https://phab.mercurial-scm.org/D2239

narrowmerge: iterate over a copy of dict items so we can mutate the dict Differential Revision: https://phab.mercurial-scm.org/D2238

contrib: ban $RANDOM using check-code Differential Revision: https://phab.mercurial-scm.org/D2237

tests: remove use of bashism $RANDOM to fix test-narrow.t on non-bash shells This test passed on our workstations, including on Macs, so we never noticed, but it fails on both the Linux and FreeBSD buildbots. Today I learned about a bash feature, wherein $RANDOM gives a random int every time you read it. check-code rule to follow. Differential Revision: https://phab.mercurial-scm.org/D2236

narrow: fix for getting the username when running http server Differential Revision: https://phab.mercurial-scm.org/D2231

tests: remove code to support Mercurial 4.3 Now that narrow lives in core, we don't need the legacy support. Differential Revision: https://phab.mercurial-scm.org/D2202

narrowspec: move module into core Having support for parsing the narrow specification in core is necessary for moving many other parts of narrow to core. We do still want to harmonize the narrow spec with the sparse spec. And the format needs to be documented. But this shouldn't hold up the code moving to core. Differential Revision: https://phab.mercurial-scm.org/D2201

hg: move share._getsrcrepo into core The fact we were calling this from extensions was a sign that it should live in core. We were also able to remove some extra attribute aliases from the share extension. Differential Revision: https://phab.mercurial-scm.org/D2200

tests: glob over line number Differential Revision: https://phab.mercurial-scm.org/D2199

tests: remove references to bundle2-exp config option This option was deleted a while ago. We don't even alias it in core. Differential Revision: https://phab.mercurial-scm.org/D2198

py3: whitelist another six passing tests Differential Revision: https://phab.mercurial-scm.org/D2230

narrowrepo: filter() is a generator on py3, wrap in list() Was at the top of Python 3 exceptions. Differential Revision: https://phab.mercurial-scm.org/D2229

narrowcommands: use pycompat.{bytes,str}kwargs Differential Revision: https://phab.mercurial-scm.org/D2228

debugcommands: mergestate version is an int, use %d on it Differential Revision: https://phab.mercurial-scm.org/D2227

progress: use %d to format ints instead of %s Due to behavioral changes between '% Ns' and '% Nd' this has some unfortunate extra dancing. I'm not sure of a better way to solve this problem. Differential Revision: https://phab.mercurial-scm.org/D2226

progress: determine padding width portably Differential Revision: https://phab.mercurial-scm.org/D2225

branchmap: wrap builtin exception in bytes for logging Differential Revision: https://phab.mercurial-scm.org/D2224

remotenames: add three new revsets related to remotenames This patch adds three new revsets 'remotenames', 'remotebookmarks' and 'remotebranches' which will return changesets which have remotenames, remotebookmarks and remotebranches on them respectively. The remotenames revset exist in hgremotenames extension and is moved from there whereas rest of the two revsets are introduced in this patch. hgremotenames: https://bitbucket.org/seanfarley/hgremotenames Differential Revision: https://phab.mercurial-scm.org/D2002

convert: fix line ending of mapfile and commit.desc file Follows up 42a393ea56d2. CRLF vs LF doesn't really matter as we do strip() or rstrip() on read, but mixing them isn't nice. So let's restore the old behavior. I don't know whether CVS/Root, CVS/Repository, and ~/.cvspass are written in native line ending, so I leave them read as binary files.

py3: port string formatting `skipped` is a set of integer revs here. So cast integers to strings using %d. None doesn't cast to an empty string on Python 3. So use '' explicitly. Differential Revision: https://phab.mercurial-scm.org/D2146

bdiff: write a native version of splitnewlines ./hg perfunidiff mercurial/manifest.py 0 --count 500 --profile before: ! wall 0.309280 comb 0.350000 user 0.290000 sys 0.060000 (best of 32) ./hg perfunidiff mercurial/manifest.py 0 --count 500 --profile after: ! wall 0.241572 comb 0.260000 user 0.240000 sys 0.020000 (best of 39) so it's about 20% faster. I hate Python. I wish we could usefully write this in Rust, but it doesn't look like that's realistic without using the cpython crate, which I'd still like to avoid. Differential Revision: https://phab.mercurial-scm.org/D1973

label: enforce the lack of leading or trailing white space In practice, all commands create label are currently striping external white space. Let us enforce this logic at a lower level before starting to rely on it elsewhere.

buildrpm: bump bundled docutils version to 0.14 when building for centos{5,6} when building rpm packages for centos 5 and 6, we bundle a mercurial-specific version of docutils in /opt/python-hg/lib/python2.7/site-packages/docutils Let's bump the embedded docutils version from 0.12 (released in 2014) to 0.14 (latest as of today)

buildrpm: bump bundled python version to 2.7.14 when building for centos{5,6} when building rpm packages for centos 5 and 6, we bundle a mercurial-specific version of python 2.7 in /opt/python-hg Let's bump the embedded python version from 2.7.10 (released in 2015) to 2.7.14 (latest as of today)

merge: invoke scmutil.fileprefetchhooks() prior to applying updates This moves the file list calculation into core, so other extensions don't need to duplicate it.

revert: drop the remnant of the prefetchfiles hook

cat: migrate to the fileprefetch callback mechanism

archive: migrate to the fileprefetch callback mechanism

lfs: migrate to the fileprefetch callback mechanism

cmdutil: convert the prefetchfiles() hook to a callback mechanism (API) Yuya suggested a list of callbacks instead of function wrapping. This means that one extension can't block another from processing the list. .. api:: File prefetching is now handled by registering a callback with scmutil.fileprefetchhooks.

test-narrow: partially stabilize on Windows test-narrow-strip.t is still broken. I don't see any feature condititionals, so it might be an actual problem.

py3: whitelist test-convert-clonebranches.t Differential Revision: https://phab.mercurial-scm.org/D2223

convert: make hg sha1 regex consistently be a bytes Differential Revision: https://phab.mercurial-scm.org/D2222

convcmd: pass encoding name as a sysstr Differential Revision: https://phab.mercurial-scm.org/D2221

convert: open all files in binary mode Differential Revision: https://phab.mercurial-scm.org/D2220

py3: whitelist another 11 passing tests Differential Revision: https://phab.mercurial-scm.org/D2213

util: convert traceback-related sysstrs to sysbytes in getstackframes With this change, test-devel-warnings.t passes except for differences in how ProgrammingError is formatted and one mysterious traceback inside of transaction.__del__. Differential Revision: https://phab.mercurial-scm.org/D2212

util: format line number of stack trace using %d Required on Python 3, functional on Python 2. Differential Revision: https://phab.mercurial-scm.org/D2211

util: call warnings.warn() with a sysstr in nouideprecwarn This preserves nouideprecwarn wanting a bytes, which is consistent with the rest of hg. Differential Revision: https://phab.mercurial-scm.org/D2210

ui: convert stack traces to sysbytes before logging They're coming back as unicodes, so sysbytes seems like the best we can do. This is like D2171 and D2172, but those fail on Python 2.7.5 (seriously!), so this is my version of the same change. I actually wrote this before reviewing those, then discarded it, then came back to it after finding out 2.7.5 is a silly place. Differential Revision: https://phab.mercurial-scm.org/D2209

ui: use pycompat.bytestr() to get a bytes-repr of config default We don't know what type we'll get here, so we need something that'll behave like a repr without returning a unicode. Differential Revision: https://phab.mercurial-scm.org/D2208

tests: add tons of b prefixes in test-devel-warnings.t # skip-blame because it's just b prefixes Differential Revision: https://phab.mercurial-scm.org/D2207

py3: convert traceback representation to bytes when logging Python's APIs should return a system string. We want to use bytes for logging. So convert the traceback to bytes before sending it to the logger. Differential Revision: https://phab.mercurial-scm.org/D2176

py3: convert context to bytes instead of str This yields identical behavior on Python 2 and 3. Differential Revision: https://phab.mercurial-scm.org/D2174

py3: add missing b'' literal to sshprotoext.py Everywhere else in this file uses b''. I must have missed a line when I wrote this code a few days ago. # skip-blame b prefix Differential Revision: https://phab.mercurial-scm.org/D2173

py3: make dummyssh compatible with Python 3 Required encoding a str to bytes and writing bytes to a file descriptor. Differential Revision: https://phab.mercurial-scm.org/D2170

mail: import email.utils not email.Utils email.Utils was renamed to email.utils in Python 2.5. But it appears at some point before Python 2.7 (possibly 2.5) email.Utils was aliased to the new home. So email.utils should be safe for all versions of Python we support. Differential Revision: https://phab.mercurial-scm.org/D2168

py3: cast character set to bytes The returned value from this function is eventually fed into email.charset.Charset.__init__. However, the standard library appears to handle both string types in Python 2 and 3. So there shouldn't be any problem returning a bytes here. Differential Revision: https://phab.mercurial-scm.org/D2167

py3: cast decode() argument to system string The actual types here may be mixed due to string literals and variables from other modules. So a cast is necessary. Differential Revision: https://phab.mercurial-scm.org/D2166

py3: avoid changing dictionary during iteration dict.items() and friends are iterators/views in Python 3. You aren't allowed to mutate the underlying dictionary when iterating on these views. So iterate over a copy of things. Differential Revision: https://phab.mercurial-scm.org/D2164

py3: use raw string for key in **kwargs Otherwise Python 3 is unhappy. We only set a single key here. So I don't think pycompat.strkwargs() is necessary. Differential Revision: https://phab.mercurial-scm.org/D2163

py3: preserve chunks as an iterable of bytes patch.diffstat() returns a bytes. util.chunkbuffer() expects an iterable of bytes. Feeding it a single bytes instance on Python 3 will iterate the characters as integers. This blows up util.chunkbuffer. Differential Revision: https://phab.mercurial-scm.org/D2162

py3: use b'' for changegroup version literals Test extensions aren't currently loaded with our custom module importer. # skip-blame b prefix Differential Revision: https://phab.mercurial-scm.org/D2161

py3: use b'' in inline extension # skip-blame b prefix Differential Revision: https://phab.mercurial-scm.org/D2160

py3: use hex(hasher.digest()) .hexdigest() returns a system str. .digest() consistently returns bytes. Differential Revision: https://phab.mercurial-scm.org/D2159

py3: use string for "close" value in commit extras The extras dict on commits has string keys and values. Before this commit, we passed "close" as an integer, which Python 2 gladly coerced to a string during a %s formatting in the bowels of changelog.py. Differential Revision: https://phab.mercurial-scm.org/D2157

py3: catch TypeError during template operations Two places in this code Python 3 changed from raising ValueError to TypeError. So catch the addition exceptions. IMO this code might be better off performing type sniffing. But I'm not sure the implications of changing that. Differential Revision: https://phab.mercurial-scm.org/D2156

py3: use bytes literals for test extension These extensions don't get run through our custom module importer. It's easy enough to make them dual compatible with Python 2 and 3. # skip-blame b prefixes Differential Revision: https://phab.mercurial-scm.org/D2155

py3: use system strings when calling __import__ We must pass the native str type when importing. Differential Revision: https://phab.mercurial-scm.org/D2154

py3: open patches.queue in binary mode And switch to using the context manager form of open() while we're here. Differential Revision: https://phab.mercurial-scm.org/D2153

py3: compare against bytes instead of str Differential Revision: https://phab.mercurial-scm.org/D2152

py3: use b'' in mockblackbox.py This unmasks a ton of other TypeError in blackbox code. # skip-blame since it's just a b prefix Differential Revision: https://phab.mercurial-scm.org/D2175

narrow: add a TODO document These are things that are bigger than we want to handle right now, but are pretty important to get narrowing to be non-experimental. Differential Revision: https://phab.mercurial-scm.org/D2196

tests: use `hello` not `capabilities` over ssh Differential Revision: https://phab.mercurial-scm.org/D2195

narrowwirepeer: add TODO about how we add wireproto args to unbundle :( Differential Revision: https://phab.mercurial-scm.org/D2194

narrowwirepeer: rename expandnarrow capability to exp-expandnarrow The expandnarrow functionality lets a client have a shorthand (for Google it's a reference to a checked-in file) for a set of includes and excludes. For testing we should probably implement a simple version of that functionality here. For now, rename the capability so we don't burn the good name in the future if we need to change behavior. It's plausible that this functionality should be dropped from the narrowhg we ship long-term, but I'm dubious as it seems pretty likely other organizations will want similar shorthands for commonly-used subsets of their trees. Differential Revision: https://phab.mercurial-scm.org/D2193

narrow: make restrictpatterns a little more idiomatic I'm not sure why invalid_includes was an out-param, but it doesn't look like there's any reason for it to be now. Differential Revision: https://phab.mercurial-scm.org/D2192

narrowspec: consistently use set() to copy sets Differential Revision: https://phab.mercurial-scm.org/D2191

narrowspec: document constraints when validating patterns Differential Revision: https://phab.mercurial-scm.org/D2190

narrowrevlog: add a TODO around remotefilelog moving to core We should clean this up considerably when remotefilelog lands, which we expect to try and accomplish in the near-ish future. Differential Revision: https://phab.mercurial-scm.org/D2189

narrowrevlog: add what little I can remember about rename filtering Differential Revision: https://phab.mercurial-scm.org/D2188

manifest: clean up dirlog() to take a d parameter to avoid shadowing dir() Differential Revision: https://phab.mercurial-scm.org/D2187

narrowrevlog: replace AssertionError with ProgrammingError Differential Revision: https://phab.mercurial-scm.org/D2186

narrowrevlog: document excludeddir class and friends Differential Revision: https://phab.mercurial-scm.org/D2185

narrowrepo: add docstring for narrowpats Differential Revision: https://phab.mercurial-scm.org/D2184

narrowtemplates: update to use registrar mechanism It's worth considering at this point if we should just move the two ellipsis functions to core tagged experimental or something, since they're not really narrowing-specific... Differential Revision: https://phab.mercurial-scm.org/D2009

narrow: move from ELLIPSIS_NODE_FLAG to revlog.REVIDX_ELLIPSIS We should have been using this value all along. Sigh. Differential Revision: https://phab.mercurial-scm.org/D2008

narrowrepo: make repo requirement include the string 'experimental' We can just move to "narrow" or similar when we finalize the format. I'm not sure what the migration process from one requirement to another should look like, but we're about to cross that bridge at Google once this change lands, so hopefully we'll know soon. Differential Revision: https://phab.mercurial-scm.org/D2007

narrow: mark requirement as a constant Differential Revision: https://phab.mercurial-scm.org/D2006

narrowbundle2: mark most constants as module-private This became obvious when preparing the previous change, so there's no reason to not just do it now while it's easy. Differential Revision: https://phab.mercurial-scm.org/D2005

narrowbundle2: make constants ALLCAPS to be a bit more readable Differential Revision: https://phab.mercurial-scm.org/D2004

narrow: remove old version-checking logic and declare internal Differential Revision: https://phab.mercurial-scm.org/D1980

tests: fold narrow treemanifest tests into main test file using testcases These tests predate the testcases functionality in run-tests.py, so it was never done in narrowhg. Differential Revision: https://phab.mercurial-scm.org/D1979

narrow: assume addflagprocessor will always exist on revlog module Differential Revision: https://phab.mercurial-scm.org/D1978

narrow: this code should assume REVIDX_FLAGS_ORDER exists Differential Revision: https://phab.mercurial-scm.org/D1977

narrow: drop legacy support for getsubsetraw Differential Revision: https://phab.mercurial-scm.org/D1976

narrow: remove support for old match modules Differential Revision: https://phab.mercurial-scm.org/D1975

narrow: import experimental extension from narrowhg revision cb51d673e9c5 Adjustments: * renamed src to hgext/narrow * marked extension experimental * added correct copyright header where it was missing * updated hgrc extension enable line in library.sh * renamed library.sh to narrow-library.sh * dropped all files from repo root as they're not interesting * dropped test-pyflakes.t, test-check-code.t and test-check-py3-compat.t * renamed remaining tests to all be test-narrow-* when they didn't already * fixed test-narrow-expanddirstate.t to refer to narrow and not narrowhg * fixed tests that wanted `update -C .` instead of `merge --abort` * corrected a two-space indent in narrowspec.py * added a missing _() in narrowcommands.py * fixed imports to pass the import checker * narrow only adds its --include and --exclude to clone if sparse isn't enabled to avoid breaking test-duplicateoptions.py. This is a kludge, and we'll need to come up with a better solution in the future. These were more or less the minimum to import something that would pass tests and not create a bunch of files we'll never use. Changes I intend to make as followups: * rework the test-narrow-*-tree.t tests to use the new testcases functionality in run-tests.py * remove lots of monkeypatches of core things Differential Revision: https://phab.mercurial-scm.org/D1974

python3: whitelist another four passing tests Differential Revision: https://phab.mercurial-scm.org/D2197

wireprotoserver: define and use parse_qs from urllib The cgi module is deprecated since Python 2.6. Let's replace uses of it in wireprotoserver with a similar function from urllib. Differential Revision: https://phab.mercurial-scm.org/D2094

tests: add tests for sending recognized command before handshake Rounding out our test coverage for the SSH server. Differential Revision: https://phab.mercurial-scm.org/D2093

wireprotoserver: add version to SSH protocol names (API) We recently introduced version 2 of the SSH protocol. Like we did for the peer, we will need to differentiate version 1 and 2 of the server. So, we add version components to the advertised name of the protocol handler. Nothing in core looks for the old value. But an extension may. .. api:: SSH protocol handler now advertises its name internally as "ssh-v1" instead of "ssh." Differential Revision: https://phab.mercurial-scm.org/D2092

wireproto: introduce type for raw byte responses (API) Right now we simply return a str/bytes instance for simple responses. I want all wire protocol response types to be strongly typed. So let's invent and use a type for raw bytes responses. .. api:: Wire protocol command handlers now return a wireprototypes.bytesresponse instead of a raw bytes instance. Protocol handlers will continue handling bytes instances. However, any extensions wrapping wire protocol commands will need to handle the new type. Differential Revision: https://phab.mercurial-scm.org/D2089

wireprototypes: move wire protocol response types to new module We'll be introducing more types as part of wire protocol version 2. These types are shared between the command handling code (in wireproto.py) and the protocol/transport code in wireprotoserver.py. So they need to go in a new module to prevent a cycle. The types are aliased into the wireproto module, so API compatibility is preserved. Differential Revision: https://phab.mercurial-scm.org/D2088