comparison mercurial/streamclone.py @ 29919:519a02267f90

streamclone: clear caches after writing changes into files for visibility

Before this patch, streamclone-ed changes are invisible via @filecache
properties to in-process procedures before closing transaction
(e.g. pretxnclose python hook), if corresponded property is cached
before consumev1(). Strictly speaking, caching should occur inside
(store) lock for transaction.

repo.invalidate() after closing transaction is too late to force
@filecache properties to be reloaded from changed files at next
access.

For visibility of streamclone-ed changes to in-process procedures
before closing transaction, this patch clears caches just after
writing changes into files.

BTW, regardless of changing in this patch, clearing cached properties
in consumev1() causes inconsistency, if (1) transaction is started and
(2) any @filecache property is changed before consumev1().

This patch also adds the comment to fix this (potential) inconsistency
in the future.

29918:d9c49138ab93

        repo.ui.status(_('%d files to transfer, %s of data\n') %
                       (filecount, util.bytecount(bytecount)))
        handled_bytes = 0
        repo.ui.progress(_('clone'), 0, total=bytecount, unit=_('bytes'))
        start = time.time()

        with repo.transaction('clone'):
            with repo.svfs.backgroundclosing(repo.ui, expectedcount=filecount):
                for i in xrange(filecount):
                    # XXX doesn't support '\n' or '\r' in filenames

                            handled_bytes += len(chunk)
                            repo.ui.progress(_('clone'), handled_bytes,
                                             total=bytecount, unit=_('bytes'))
                            ofp.write(chunk)

        # Writing straight to files circumvented the inmemory caches
        repo.invalidate(clearfilecache=True)

        elapsed = time.time() - start
        if elapsed <= 0:
            elapsed = 0.001
        repo.ui.progress(_('clone'), None)

29919:519a02267f90

        repo.ui.status(_('%d files to transfer, %s of data\n') %
                       (filecount, util.bytecount(bytecount)))
        handled_bytes = 0
        repo.ui.progress(_('clone'), 0, total=bytecount, unit=_('bytes'))
        start = time.time()

        # TODO: get rid of (potential) inconsistency
        #
        # If transaction is started and any @filecache property is
        # changed at this point, it causes inconsistency between
        # in-memory cached property and streamclone-ed file on the
        # disk. Nested transaction prevents transaction scope "clone"
        # below from writing in-memory changes out at the end of it,
        # even though in-memory changes are discarded at the end of it
        # regardless of transaction nesting.
        #
        # But transaction nesting can't be simply prohibited, because
        # nesting occurs also in ordinary case (e.g. enabling
        # clonebundles).

        with repo.transaction('clone'):
            with repo.svfs.backgroundclosing(repo.ui, expectedcount=filecount):
                for i in xrange(filecount):
                    # XXX doesn't support '\n' or '\r' in filenames

                            handled_bytes += len(chunk)
                            repo.ui.progress(_('clone'), handled_bytes,
                                             total=bytecount, unit=_('bytes'))
                            ofp.write(chunk)

            # force @filecache properties to be reloaded from
            # streamclone-ed file at next access
            repo.invalidate(clearfilecache=True)

        elapsed = time.time() - start
        if elapsed <= 0:
            elapsed = 0.001
        repo.ui.progress(_('clone'), None)