hgext/infinitepush/bundleparts.py

infinitepush: move the extension to core from fb-hgext

This patch moves the infinitepush extension from fb-hgext to core. The
extension is used to store incoming bundles during a push in bundlestore rather
than applying them to the revlog.

The extension was copied from the repository revision at
f27f094e91553d3cae5167c0b1c42ae940f888d5 and following changes were made:

  * added `from __future__ import absolute_import` where missing
  * fixed module imports to follow the core style
  * minor fixes for test-check-code.t
  * registered the configs
  * adding the testedwith value to match core's convention
  * removed double newlines to make test-check-commit.t happy
  * added one line doc about extension and marked it as experimental

Only one test file test-infinitepush-bundlestore.t is moved to core and
following changes are made to file:

  * remove dependency of library.sh
  * split the tests into two tests i.e. test-infinitepush.t and
    test-infinitepush-bundlestore.t
  * removed testing related to other facebook's extensions pushrebase, inhibit,
    fbamend

library-infinitepush.sh is also copied from fb-hgext from the same revision and
following changes are made:

  * change the path to infinitepush extension as it's in core with this patch
  * removed sql handling from the file as we are not testing that initially

Currently at this revision, test-check-module-imports.t does not pass as there
is import of a module from fb/hgext in one the of the file which will be removed
in the next patch.

This extension right now has a lot of things which we don't require in core like
`--to`, `--create` flags to `hg bookmark`, logic related to remotenames
extension and another facebook's extensions, custom bundle2parts which can be
prevented by using bookmarks bundle part and also logic related to sql store
which is probably we don't want initially.

The next patches in this series will remove all the unwanted and unrequired
things from the extension and will make this a nice one.

The end goal is to have a very lighweight extension with no or very less
wrapping on the client side.

Differential Revision: https://phab.mercurial-scm.org/D2096

# Copyright 2017 Facebook, Inc.
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.

from __future__ import absolute_import

from mercurial.i18n import _

from mercurial import (
    bundle2,
    changegroup,
    error,
    extensions,
    revsetlang,
    util,
)

from . import common

encodebookmarks = common.encodebookmarks
isremotebooksenabled = common.isremotebooksenabled

scratchbranchparttype = 'b2x:infinitepush'
scratchbookmarksparttype = 'b2x:infinitepushscratchbookmarks'

def getscratchbranchparts(repo, peer, outgoing, confignonforwardmove,
                         ui, bookmark, create):
    if not outgoing.missing:
        raise error.Abort(_('no commits to push'))

    if scratchbranchparttype not in bundle2.bundle2caps(peer):
        raise error.Abort(_('no server support for %r') % scratchbranchparttype)

    _validaterevset(repo, revsetlang.formatspec('%ln', outgoing.missing),
                    bookmark)

    supportedversions = changegroup.supportedoutgoingversions(repo)
    # Explicitly avoid using '01' changegroup version in infinitepush to
    # support general delta
    supportedversions.discard('01')
    cgversion = min(supportedversions)
    _handlelfs(repo, outgoing.missing)
    cg = changegroup.makestream(repo, outgoing, cgversion, 'push')

    params = {}
    params['cgversion'] = cgversion
    if bookmark:
        params['bookmark'] = bookmark
        # 'prevbooknode' is necessary for pushkey reply part
        params['bookprevnode'] = ''
        if bookmark in repo:
            params['bookprevnode'] = repo[bookmark].hex()
        if create:
            params['create'] = '1'
    if confignonforwardmove:
        params['force'] = '1'

    # Do not send pushback bundle2 part with bookmarks if remotenames extension
    # is enabled. It will be handled manually in `_push()`
    if not isremotebooksenabled(ui):
        params['pushbackbookmarks'] = '1'

    parts = []

    # .upper() marks this as a mandatory part: server will abort if there's no
    #  handler
    parts.append(bundle2.bundlepart(
        scratchbranchparttype.upper(),
        advisoryparams=params.iteritems(),
        data=cg))

    try:
        treemod = extensions.find('treemanifest')
        mfnodes = []
        for node in outgoing.missing:
            mfnodes.append(('', repo[node].manifestnode()))

        # Only include the tree parts if they all exist
        if not repo.manifestlog.datastore.getmissing(mfnodes):
            parts.append(treemod.createtreepackpart(
                repo, outgoing, treemod.TREEGROUP_PARTTYPE2))
    except KeyError:
        pass

    return parts

def getscratchbookmarkspart(peer, bookmarks):
    if scratchbookmarksparttype not in bundle2.bundle2caps(peer):
        raise error.Abort(
            _('no server support for %r') % scratchbookmarksparttype)

    return bundle2.bundlepart(
        scratchbookmarksparttype.upper(),
        data=encodebookmarks(bookmarks))

def _validaterevset(repo, revset, bookmark):
    """Abort if the revs to be pushed aren't valid for a scratch branch."""
    if not repo.revs(revset):
        raise error.Abort(_('nothing to push'))
    if bookmark:
        # Allow bundle with many heads only if no bookmark is specified
        heads = repo.revs('heads(%r)', revset)
        if len(heads) > 1:
            raise error.Abort(
                _('cannot push more than one head to a scratch branch'))

def _handlelfs(repo, missing):
    '''Special case if lfs is enabled

    If lfs is enabled then we need to call prepush hook
    to make sure large files are uploaded to lfs
    '''
    try:
        lfsmod = extensions.find('lfs')
        lfsmod.wrapper.uploadblobsfromrevs(repo, missing)
    except KeyError:
        # Ignore if lfs extension is not enabled
        return

class copiedpart(object):
    """a copy of unbundlepart content that can be consumed later"""

    def __init__(self, part):
        # copy "public properties"
        self.type = part.type
        self.id = part.id
        self.mandatory = part.mandatory
        self.mandatoryparams = part.mandatoryparams
        self.advisoryparams = part.advisoryparams
        self.params = part.params
        self.mandatorykeys = part.mandatorykeys
        # copy the buffer
        self._io = util.stringio(part.read())

    def consume(self):
        return

    def read(self, size=None):
        if size is None:
            return self._io.read()
        else:
            return self._io.read(size)