manifest: add manifestdict.hasdir() method Allows for alternative implementations of manifestdict to decide if a directory exists in whatever way is most optimal.

manifest: have context's dirs() call its manifest's dirs() This lets the context's dirs() method be agnostic towards any alternate manifest implementations.

manifest: add dirs() to manifestdict Manifests should have a method of accessing its own dirs, not just the context that references the manifest. This makes it easier for other optimized versions of manifests to compute their own dirs in the most efficient way.

formatter: convert None to json null It will be used by "annotate" command to represent the workingctx revision.

localrepo: extend "changeid in repo" to return True for workingctx revision This is necessary to implement "wc" symbol for workingctx, that will be used as follows: $ hg annotate -r wc FILE In principle, "rev in repo" should be True if "repo[rev]" can return a context object. But when it was implemented by ea3acaae25bb, lookup() had a long logic to map all sorts of changeids to nodes, and "None in repo" did crash because lookup() could not accept None. So I assume that the case of changeid=None was not considered. Now "None in repo" doesn't crash, it should be True for workingctx revision. Behavior of "changeid in repo": revision "null" existing rev None (workingctx) ---------- ------ ------------ ----------------- original* True True TypeError current True True False this patch True True True (*original: ea3acaae25bb)

merge with stable

crecord: fix another underbar

crecord: fiddle with vertical whitespace

crecord: fix underbar style for orig_stdout

crecord: use encoding.ucolwidth

crecord: more import style

crecord: fix some import style

manifest.c: ensure realloc_if_full() returns 1 or 0 This fixes an MSVC 2008 warning that I don't see with gcc 4.6.3-2: warning C4047: 'return' : 'bool' differs in levels of indirection from 'line *' More importantly, the truncation from pointer to 'unsigned char' would have returned 0 if self->lines pointed to an address divisible by 0xFF, which causes find_lines() to return MANIFEST_OOM. I was able to cause this to happen in a trivial program with the gcc compiler.

record: change recording function to work with crecord

record: add crecord's ui logic to core Code adapted from https://bitbucket.org/edgimar/crecord/src to respect coding convention an record's interface

record: change interface of dorecord to accept new filters This makes it easier to add different filtering logic (record /crecord ...)

record: remove duplicated tests Since the record and commit -i commands are identical we simplify record's test to just cover the help message and minimal smoke testing.

record: make record use commit -i

style: kill ersatz if-else ternary operators Although Python supports `X = Y if COND else Z`, this was only introduced in Python 2.5. Since we have to support Python 2.4, it was a very common thing to write instead `X = COND and Y or Z`, which is a bit obscure at a glance. It requires some intricate knowledge of Python to understand how to parse these one-liners. We change instead all of these one-liners to 4-liners. This was executed with the following perlism: find -name "*.py" -exec perl -pi -e 's,(\s*)([\.\w]+) = \(?(\S+)\s+and\s+(\S*)\)?\s+or\s+(\S*)$,$1if $3:\n$1 $2 = $4\n$1else:\n$1 $2 = $5,' {} \; I tweaked the following cases from the automatic Perl output: prev = (parents and parents[0]) or nullid port = (use_ssl and 443 or 80) cwd = (pats and repo.getcwd()) or '' rename = fctx and webutil.renamelink(fctx) or [] ctx = fctx and fctx or ctx self.base = (mapfile and os.path.dirname(mapfile)) or '' I also added some newlines wherever they seemd appropriate for readability There are probably a few ersatz ternary operators still in the code somewhere, lurking away from the power of a simple regex.

cvsps: use a different tiebreaker to avoid flaky test After adding some sneaky debug printing[0], I determined that this test flaked when a CVS commit containing two files starts too close to the end of a second, thus putting file "a" in one second and "b/c" in the following second. The secondary sort key meant that these changes sorted in a different order when the timestamps were different than they did when they matched. As far as I can tell, CVS walks through the files in a stable order, so by sorting on the filenames in cvsps we'll get stable output. It's fine for us to switch from sorting on the branchpoint as a secondary key because this was already the point when we didn't care, and we're just trying to break ties in a stable way. It's unclear to be if having the branchpoint present matters anymore, but it doesn't really hurt to leave it. With this change in place, I was able to run test-convert-cvs over 650 times in a row without a failure. test-convert-cvcs-synthetic.t appears to still be flaky, but I don't think it's *worse* than it was before - just not better (I observed one flaky failure in 200 runs on that test). 0: The helpful debug hack ended up being this, in case it's useful to future flaky test assassins: --- a/hgext/convert/cvsps.py +++ b/hgext/convert/cvsps.py @@ -854,6 +854,8 @@ def debugcvsps(ui, *args, **opts): ui.write(('Branch: %s\n' % (cs.branch or 'HEAD'))) ui.write(('Tag%s: %s \n' % (['', 's'][len(cs.tags) > 1], ','.join(cs.tags) or '(none)'))) + if cs.comment == 'ci1' and (cs.id == 6) == bool(cs.branchpoints): + ui.write('raw timestamp %r\n' % (cs.date,)) if cs.branchpoints: ui.write(('Branchpoints: %s \n') % ', '.join(sorted(cs.branchpoints)))

templates: fix "log -q" output of default style It was changed at 0ded0f0b1c04 unintentionally due to name conflicts.

merge with stable

subrepo: replace 'ctx._repo' with 'ctx.repo()'

files: replace 'ctx._repo' with 'ctx.repo()'

context: add a repo accessor There are 29 instances of 'ctx._repo' in the code, so make the ability to access more official.

test-histedit-edit.t: demonstrate qnew fails during a histedit (issue4366) This was accidentally fixed by other work, but given that it's been broken in the past, I'd like to have a test defending us against regressions in this area, especially as we add more functionality to histedit.

lazymanifest: make __iter__ generate filenames, not 3-tuples The _lazymanifest type(s) behave very much like a sorted dict with filenames as keys and (nodeid, flags) as values. It therefore seems surprising that its __iter__ generates 3-tuples of (path, nodeid, flags). Let's make it match dict's behavior of generating the keys instead, and add a new iterentries method for the 3-tuples. With this change, the "x" in "if x in lm" and "for x in lm" now have the same type (a filename string).

lazymanifest: fix pure hg iterkeys() I broke pure hg when I just added iterkeys() to the native version in 2b7ab29627fd. I forgot to make the pure version sorted. Fix it.

hgweb: prevent loading style map from directories other than specified paths A style name should not contain "/", "\", "." and "..". Otherwise, templates could be loaded from outside of the specified templates directory by invalid ?style= parameter. hgweb should not allow such requests. This change means subdir/name is also rejected.

lazymanifest: add iterkeys() method So we don't have to iteratate over (path, node, flags) tuples only to throw away the node and flags.

lazymanifest: extract function for iterating to next line This will soon be reused by keys iterator.

lazymanifest: fail if path or hash strings cannot be created While generating (path, hash, flags), we fail if flags cannot be created. We should also fail if path or hash cannot be created.

manifest: rewrite find(node, f) in terms of read(node) Since find() now always works with a full manifest, we can simplify by calling read() to give us that manifest. That way, we also populate the manifest cache. However, now that we no longer parse the manifest text into a Python type (thanks, lazymanifest/Augie), the cost of parsing (scanning for newlines, really) is small enough that it seems generally drowned by revlog reading.

ssl: load CA certificates from system's store by default on Python 2.7.9 This will make it easy to manage in-house CA certificates, which are often used in corporate environment and installed into the Windows' certs store. Unlike Apple python, the dummycert trick isn't necessary on Python 2.7.9. The default web.cacerts will be set as follows: environment web.cacerts behavior ------------- ----------- ----------------------------------------- Apple Python dummycert fall back to system's store Python 2.7.8 '!' never use CA certs (show warning instead) Python 2.7.9+ None load CA certs from system's store

ssl: set explicit symbol "!" to web.cacerts to disable SSL verification (BC) The next patch will enable verification by using the system's CA store if possible, which means we would have to distinguish None (=use default) from '' (=--insecure). This smells bug-prone and provides no way to override web.cacerts to forcibly use the system's store by --config argument. This patch changes the meaning of web.cacerts as follows: value behavior ------- --------------------------------------- None/'' use default '!' never use CA certs (set by --insecure) <path> verify by the specified CA certificates Values other than <path> are for internal use and therefore undocumented.

test-https: enable dummycert test only if Apple python is used (issue4500) The dummycert trick works only if Python is linked to Apple's patched OpenSSL.

ssl: extract function that returns dummycert path on Apple python This function will be the condition to switch DISABLEOSXDUMMYCERT in test-https.t.

largefiles: don't create chain of __contains__ calls Matt Harbison pointed out that my recent 2720f967a7b1 might cause __contains__ to continously get replaced by another version that calls itself, since the manifest instance returned by the super method is always the same instance due to @propertycache. He also suggested replacing the class instead, as is done with the context class in the surrounding code. Do so.

lazymanifest: don't depend on printf's 'hh' format to work Where we convert a 20-byte binary to a 40-byte hex string in lazymanifest_setitem(), we use sprintf("%02hhx", hash[i]). As Matt Harbison found out, 'hh' seems to be ignored on some platforms (Visual Studio?). If char is signed on such platforms, the value gets sign-extended as it gets promoted into an int when passed into the variadic sprintf(). The resulting integer value will then be printed with leading f's (14 of them on 64-bit systems), since the '2' in the format string indicates only minimum number of characters. This is both incorrect and runs the risk of writing outside of allocated memory (as Matt reported). To fix, let's cast the value to unsigned char before passing it to sprintf(). Also drop the poorly supported 'hh' formatting that now becomes unnecessary.

bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.

hook: add a generic hook right before we commit a transaction We are adding a 'txnclose' hook that will be run right before a transaction is closed. Hooks running at that time will have access to the full transaction content through both 'hookargs' content and on-disk reading. They will be able to abort the transaction.

transaction: add a validation stage The 'transaction' object can now be fed a 'validator' function. This function will be run right before the transaction is closed to validate its content. The target usage is hooks. The validation function is expected to raise an exception when it wants to abort the transaction. This only introduce the idea with a default no-op validator. Actual usage is in the next changeset.

hook: add a generic hook after transaction has been closed We are adding generic hooking for all transactions. We may have useful information about what happened during the transaction, user of the transaction should have filled the 'hookargs' dictionnary of the transaction. This hook is simple because it has no power to rollback the transaction.

hook: have a generic hook for transaction opening We are adding generic hooking for all transactions. We do not really have any useful information to include when opening the transaction but this is a useful time to allow a hook anyway. We better let people abort transaction before they happen than after multiple seconds/minutes of processing.

templater: allow piping generator-type function output to filters Template functions use "yield"s assuming that the result will be combined into a string, which means both "f -> str" and "f -> generator" should behave in the same way. Before this patch, piping generator function resulted in a cryptic error. We had to insert "|stringify" in this case. $ hg log --template '{if(author, author)|user}\n' abort: template filter 'userfilter' is not compatible with keyword '[(<function runsymbol at 0x7f5af2e8d8c0>, 'author'), (<function runsymbol at 0x7f5af2e8d8c0>, 'author')]'

record: add new tests for commit interactive (same tests as record)

record: add interactive option to the commit command

manifest: don't let find() look inside manifestdict The find() method is currently implemented by looking inside the _lm field of the manifest dict. Future manifests types (tree manifests) may not have such a field, so add a method for getting to the data instead.