hgext/convert/subversion.py
author Manuel Jacob <me@manueljacob.de>
Mon, 29 Jun 2020 15:03:36 +0200
branchstable
changeset 44831 e3b19004087a
parent 44830 cb097496138a
child 44832 e54c3cafda15
permissions -rw-r--r--
convert: correctly convert paths to UTF-8 for Subversion The previous code using encoding.tolocal() only worked by chance in these situations: * The string is ASCII: The fast path was triggered and the string was returned unmodified. * The local encoding is UTF-8: The source and target encoding is the same. * The string is not valid UTF-8 and the native encoding is ISO-8859-1: If the string doesn’t decode using UTF-8, ISO-8859-1 is tried as a fallback. During `hg convert`, the local encoding is always UTF-8. The irony is that in this case, encoding.tolocal() behaves like what someone would expect the reverse function, encoding.fromlocal(), to do. When the locale encoding is ISO-8859-15, trying to convert a SVN repo `/tmp/a€` failed before like this: file:///tmp/a%C2%A4 does not look like a Subversion repository to libsvn version 1.14.0 The correct URL is `file:///tmp/a%E2%82%AC`. Unlike previously (with the ISO-8859-1 fallback), decoding the path using the locale encoding can fail. In this case, we have to bail out, as Subversion won’t be able to do anything useful with the path.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
     1
# Subversion 1.4/1.5 Python API backend
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
     2
#
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
     3
# Copyright(C) 2007 Daniel Holth et al
28408
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
     4
from __future__ import absolute_import
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
     5
44831
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
     6
import codecs
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
     7
import locale
28408
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
     8
import os
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
     9
import re
19787
87f07f70f227 convert: move import of xml.minidom.dom to its own line for check-code
Augie Fackler <raf@durin42.com>
parents: 19468
diff changeset
    10
import xml.dom.minidom
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
    11
29205
a0939666b836 py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents: 28883
diff changeset
    12
from mercurial.i18n import _
43085
eef9a2d67051 py3: manually import pycompat.open into files that need it
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
    13
from mercurial.pycompat import open
28408
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    14
from mercurial import (
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    15
    encoding,
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    16
    error,
30528
20a42325fdef py3: use pycompat.getcwd() instead of os.getcwd()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30275
diff changeset
    17
    pycompat,
28408
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    18
    util,
31256
b023caa6dbae vfs: use 'vfs' module directly in 'hgext.convert'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31226
diff changeset
    19
    vfs as vfsmod,
28408
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    20
)
37087
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36843
diff changeset
    21
from mercurial.utils import (
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36843
diff changeset
    22
    dateutil,
37123
a8a902d7176e procutil: bulk-replace function calls to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37087
diff changeset
    23
    procutil,
37087
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36843
diff changeset
    24
    stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36843
diff changeset
    25
)
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
    26
28408
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    27
from . import common
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    28
29324
b501579147f1 py3: conditionalize cPickle import by adding in util
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29205
diff changeset
    29
pickle = util.pickle
28861
86db5cb55d46 pycompat: switch to util.stringio for py3 compat
timeless <timeless@mozdev.org>
parents: 28471
diff changeset
    30
stringio = util.stringio
16511
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
    31
propertycache = util.propertycache
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
    32
urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
    33
urlreq = util.urlreq
16511
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
    34
28408
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    35
commandline = common.commandline
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    36
commit = common.commit
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    37
converter_sink = common.converter_sink
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    38
converter_source = common.converter_source
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    39
decodeargs = common.decodeargs
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    40
encodeargs = common.encodeargs
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    41
makedatetimestamp = common.makedatetimestamp
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    42
mapfile = common.mapfile
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    43
MissingTool = common.MissingTool
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    44
NoRepo = common.NoRepo
4ac63ed377ba convert: subversion use absolute_import
timeless <timeless@mozdev.org>
parents: 27314
diff changeset
    45
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
    46
# Subversion stuff. Works best with very recent Python SVN bindings
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
    47
# e.g. SVN 1.5 or backports. Thanks to the bzr folks for enhancing
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
    48
# these bindings.
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
    49
4766
95cbb6b74790 convert: activate subversion engine
Brendan Cully <brendan@kublai.com>
parents: 4765
diff changeset
    50
try:
5010
6b2d8caf87b2 convert svn: try to extract URL from source if it is a working directory
Brendan Cully <brendan@kublai.com>
parents: 5008
diff changeset
    51
    import svn
6b2d8caf87b2 convert svn: try to extract URL from source if it is a working directory
Brendan Cully <brendan@kublai.com>
parents: 5008
diff changeset
    52
    import svn.client
4766
95cbb6b74790 convert: activate subversion engine
Brendan Cully <brendan@kublai.com>
parents: 4765
diff changeset
    53
    import svn.core
95cbb6b74790 convert: activate subversion engine
Brendan Cully <brendan@kublai.com>
parents: 4765
diff changeset
    54
    import svn.ra
95cbb6b74790 convert: activate subversion engine
Brendan Cully <brendan@kublai.com>
parents: 4765
diff changeset
    55
    import svn.delta
28459
3ea62e748a18 convert: make subversion import transport locally
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 28408
diff changeset
    56
    from . import transport
8221
f35b933044cc convert: hide svn deprecation warnings
Ronny Pfannschmidt <Ronny.Pfannschmidt@gmx.de>
parents: 8219
diff changeset
    57
    import warnings
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
    58
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
    59
    warnings.filterwarnings(
44809
4888adfb6235 py3: unbyteify arguments to warnings.filterwarnings()
Manuel Jacob <me@manueljacob.de>
parents: 44470
diff changeset
    60
        'ignore', module='svn.core', category=DeprecationWarning
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
    61
    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
    62
    svn.core.SubversionException  # trigger import to catch error
8221
f35b933044cc convert: hide svn deprecation warnings
Ronny Pfannschmidt <Ronny.Pfannschmidt@gmx.de>
parents: 8219
diff changeset
    63
4766
95cbb6b74790 convert: activate subversion engine
Brendan Cully <brendan@kublai.com>
parents: 4765
diff changeset
    64
except ImportError:
13480
69418d4525d1 convert/svn: abort operation when python bindings are not available
Azhagu Selvan SP <tamizhgeek@gmail.com>
parents: 13190
diff changeset
    65
    svn = None
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
    66
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
    67
44831
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    68
# In Subversion, paths are Unicode (encoded as UTF-8), which Subversion
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    69
# converts from / to native strings when interfacing with the OS. When passing
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    70
# paths to Subversion, we have to recode them such that it roundstrips with
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    71
# what Subversion is doing.
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    72
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    73
fsencoding = None
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    74
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    75
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    76
def init_fsencoding():
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    77
    global fsencoding, fsencoding_is_utf8
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    78
    if fsencoding is not None:
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    79
        return
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    80
    if pycompat.iswindows:
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    81
        # On Windows, filenames are Unicode, but we store them using the MBCS
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    82
        # encoding.
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    83
        fsencoding = 'mbcs'
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    84
    else:
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    85
        # This is the encoding used to convert UTF-8 back to natively-encoded
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    86
        # strings in Subversion 1.14.0 or earlier with APR 1.7.0 or earlier.
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    87
        with util.with_lc_ctype():
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    88
            fsencoding = locale.nl_langinfo(locale.CODESET) or 'ISO-8859-1'
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    89
    fsencoding = codecs.lookup(fsencoding).name
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    90
    fsencoding_is_utf8 = fsencoding == codecs.lookup('utf-8').name
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    91
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    92
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    93
def fs2svn(s):
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    94
    if fsencoding_is_utf8:
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    95
        return s
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    96
    else:
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    97
        return s.decode(fsencoding).encode('utf-8')
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    98
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
    99
7381
b965605dfb2e convert: be even more tolerant when detecting svn tags
Patrick Mezard <pmezard@gmail.com>
parents: 7184
diff changeset
   100
class SvnPathNotFound(Exception):
b965605dfb2e convert: be even more tolerant when detecting svn tags
Patrick Mezard <pmezard@gmail.com>
parents: 7184
diff changeset
   101
    pass
b965605dfb2e convert: be even more tolerant when detecting svn tags
Patrick Mezard <pmezard@gmail.com>
parents: 7184
diff changeset
   102
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   103
13690
af331f557942 convert/svn: extract revsplit() in a function
Patrick Mezard <pmezard@gmail.com>
parents: 13653
diff changeset
   104
def revsplit(rev):
20419
e61a8395c3c1 convert: make subversion revsplit more stable when meeting revisions without @
Mads Kiilerich <madski@unity3d.com>
parents: 20378
diff changeset
   105
    """Parse a revision string and return (uuid, path, revnum).
34146
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 31456
diff changeset
   106
    >>> revsplit(b'svn:a2147622-4a9f-4db4-a8d3-13562ff547b2'
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 31456
diff changeset
   107
    ...          b'/proj%20B/mytrunk/mytrunk@1')
20419
e61a8395c3c1 convert: make subversion revsplit more stable when meeting revisions without @
Mads Kiilerich <madski@unity3d.com>
parents: 20378
diff changeset
   108
    ('a2147622-4a9f-4db4-a8d3-13562ff547b2', '/proj%20B/mytrunk/mytrunk', 1)
34146
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 31456
diff changeset
   109
    >>> revsplit(b'svn:8af66a51-67f5-4354-b62c-98d67cc7be1d@1')
20419
e61a8395c3c1 convert: make subversion revsplit more stable when meeting revisions without @
Mads Kiilerich <madski@unity3d.com>
parents: 20378
diff changeset
   110
    ('', '', 1)
34146
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 31456
diff changeset
   111
    >>> revsplit(b'@7')
20419
e61a8395c3c1 convert: make subversion revsplit more stable when meeting revisions without @
Mads Kiilerich <madski@unity3d.com>
parents: 20378
diff changeset
   112
    ('', '', 7)
34146
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 31456
diff changeset
   113
    >>> revsplit(b'7')
20419
e61a8395c3c1 convert: make subversion revsplit more stable when meeting revisions without @
Mads Kiilerich <madski@unity3d.com>
parents: 20378
diff changeset
   114
    ('', '', 0)
34146
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 31456
diff changeset
   115
    >>> revsplit(b'bad')
20419
e61a8395c3c1 convert: make subversion revsplit more stable when meeting revisions without @
Mads Kiilerich <madski@unity3d.com>
parents: 20378
diff changeset
   116
    ('', '', 0)
e61a8395c3c1 convert: make subversion revsplit more stable when meeting revisions without @
Mads Kiilerich <madski@unity3d.com>
parents: 20378
diff changeset
   117
    """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   118
    parts = rev.rsplit(b'@', 1)
20419
e61a8395c3c1 convert: make subversion revsplit more stable when meeting revisions without @
Mads Kiilerich <madski@unity3d.com>
parents: 20378
diff changeset
   119
    revnum = 0
e61a8395c3c1 convert: make subversion revsplit more stable when meeting revisions without @
Mads Kiilerich <madski@unity3d.com>
parents: 20378
diff changeset
   120
    if len(parts) > 1:
e61a8395c3c1 convert: make subversion revsplit more stable when meeting revisions without @
Mads Kiilerich <madski@unity3d.com>
parents: 20378
diff changeset
   121
        revnum = int(parts[1])
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   122
    parts = parts[0].split(b'/', 1)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   123
    uuid = b''
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   124
    mod = b''
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   125
    if len(parts) > 1 and parts[0].startswith(b'svn:'):
20419
e61a8395c3c1 convert: make subversion revsplit more stable when meeting revisions without @
Mads Kiilerich <madski@unity3d.com>
parents: 20378
diff changeset
   126
        uuid = parts[0][4:]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   127
        mod = b'/' + parts[1]
20419
e61a8395c3c1 convert: make subversion revsplit more stable when meeting revisions without @
Mads Kiilerich <madski@unity3d.com>
parents: 20378
diff changeset
   128
    return uuid, mod, revnum
13690
af331f557942 convert/svn: extract revsplit() in a function
Patrick Mezard <pmezard@gmail.com>
parents: 13653
diff changeset
   129
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   130
15599
c6be93a4c378 convert/svn: fix URL quoting issue with svn 1.7
Patrick Mezard <pmezard@gmail.com>
parents: 15381
diff changeset
   131
def quote(s):
c6be93a4c378 convert/svn: fix URL quoting issue with svn 1.7
Patrick Mezard <pmezard@gmail.com>
parents: 15381
diff changeset
   132
    # As of svn 1.7, many svn calls expect "canonical" paths. In
c6be93a4c378 convert/svn: fix URL quoting issue with svn 1.7
Patrick Mezard <pmezard@gmail.com>
parents: 15381
diff changeset
   133
    # theory, we should call svn.core.*canonicalize() on all paths
c6be93a4c378 convert/svn: fix URL quoting issue with svn 1.7
Patrick Mezard <pmezard@gmail.com>
parents: 15381
diff changeset
   134
    # before passing them to the API.  Instead, we assume the base url
c6be93a4c378 convert/svn: fix URL quoting issue with svn 1.7
Patrick Mezard <pmezard@gmail.com>
parents: 15381
diff changeset
   135
    # is canonical and copy the behaviour of svn URL encoding function
c6be93a4c378 convert/svn: fix URL quoting issue with svn 1.7
Patrick Mezard <pmezard@gmail.com>
parents: 15381
diff changeset
   136
    # so we can extend it safely with new components. The "safe"
c6be93a4c378 convert/svn: fix URL quoting issue with svn 1.7
Patrick Mezard <pmezard@gmail.com>
parents: 15381
diff changeset
   137
    # characters were taken from the "svn_uri__char_validity" table in
c6be93a4c378 convert/svn: fix URL quoting issue with svn 1.7
Patrick Mezard <pmezard@gmail.com>
parents: 15381
diff changeset
   138
    # libsvn_subr/path.c.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   139
    return urlreq.quote(s, b"!$&'()*+,-./:=@_~")
15599
c6be93a4c378 convert/svn: fix URL quoting issue with svn 1.7
Patrick Mezard <pmezard@gmail.com>
parents: 15381
diff changeset
   140
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   141
5008
b6c3abdbe0eb convert: urlify svn repos if necessary.
Brendan Cully <brendan@kublai.com>
parents: 4960
diff changeset
   142
def geturl(path):
5010
6b2d8caf87b2 convert svn: try to extract URL from source if it is a working directory
Brendan Cully <brendan@kublai.com>
parents: 5008
diff changeset
   143
    try:
5020
780051cca03c convert svn: canonicalize path before calling url_from_path.
Brendan Cully <brendan@kublai.com>
parents: 5010
diff changeset
   144
        return svn.client.url_from_path(svn.core.svn_path_canonicalize(path))
28460
d6dbb0968c13 convert: fix relative import of stdlib module in subversion
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 28459
diff changeset
   145
    except svn.core.SubversionException:
15599
c6be93a4c378 convert/svn: fix URL quoting issue with svn 1.7
Patrick Mezard <pmezard@gmail.com>
parents: 15381
diff changeset
   146
        # svn.client.url_from_path() fails with local repositories
5010
6b2d8caf87b2 convert svn: try to extract URL from source if it is a working directory
Brendan Cully <brendan@kublai.com>
parents: 5008
diff changeset
   147
        pass
5008
b6c3abdbe0eb convert: urlify svn repos if necessary.
Brendan Cully <brendan@kublai.com>
parents: 4960
diff changeset
   148
    if os.path.isdir(path):
5793
68f5bf9aa582 convert: Accept local path on win32.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 5790
diff changeset
   149
        path = os.path.normpath(os.path.abspath(path))
34645
75979c8d4572 codemod: use pycompat.iswindows
Jun Wu <quark@fb.com>
parents: 34190
diff changeset
   150
        if pycompat.iswindows:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   151
            path = b'/' + util.normpath(path)
8886
0593c5b5b1f0 convert/svn: stop returning unicode revision identifiers
Patrick Mezard <pmezard@gmail.com>
parents: 8885
diff changeset
   152
        # Module URL is later compared with the repository URL returned
0593c5b5b1f0 convert/svn: stop returning unicode revision identifiers
Patrick Mezard <pmezard@gmail.com>
parents: 8885
diff changeset
   153
        # by svn API, which is UTF-8.
44831
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
   154
        path = fs2svn(path)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   155
        path = b'file://%s' % quote(path)
15599
c6be93a4c378 convert/svn: fix URL quoting issue with svn 1.7
Patrick Mezard <pmezard@gmail.com>
parents: 15381
diff changeset
   156
    return svn.core.svn_path_canonicalize(path)
5008
b6c3abdbe0eb convert: urlify svn repos if necessary.
Brendan Cully <brendan@kublai.com>
parents: 4960
diff changeset
   157
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   158
5117
c89ed1f27e78 convert: svn: add helper function for optrevs
Brendan Cully <brendan@kublai.com>
parents: 5114
diff changeset
   159
def optrev(number):
c89ed1f27e78 convert: svn: add helper function for optrevs
Brendan Cully <brendan@kublai.com>
parents: 5114
diff changeset
   160
    optrev = svn.core.svn_opt_revision_t()
c89ed1f27e78 convert: svn: add helper function for optrevs
Brendan Cully <brendan@kublai.com>
parents: 5114
diff changeset
   161
    optrev.kind = svn.core.svn_opt_revision_number
c89ed1f27e78 convert: svn: add helper function for optrevs
Brendan Cully <brendan@kublai.com>
parents: 5114
diff changeset
   162
    optrev.value.number = number
c89ed1f27e78 convert: svn: add helper function for optrevs
Brendan Cully <brendan@kublai.com>
parents: 5114
diff changeset
   163
    return optrev
c89ed1f27e78 convert: svn: add helper function for optrevs
Brendan Cully <brendan@kublai.com>
parents: 5114
diff changeset
   164
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   165
4946
e8f4e40f285a convert/subversion: work around memory leak in svn's python bindings
Bryan O'Sullivan <bos@serpentine.com>
parents: 4944
diff changeset
   166
class changedpath(object):
e8f4e40f285a convert/subversion: work around memory leak in svn's python bindings
Bryan O'Sullivan <bos@serpentine.com>
parents: 4944
diff changeset
   167
    def __init__(self, p):
e8f4e40f285a convert/subversion: work around memory leak in svn's python bindings
Bryan O'Sullivan <bos@serpentine.com>
parents: 4944
diff changeset
   168
        self.copyfrom_path = p.copyfrom_path
e8f4e40f285a convert/subversion: work around memory leak in svn's python bindings
Bryan O'Sullivan <bos@serpentine.com>
parents: 4944
diff changeset
   169
        self.copyfrom_rev = p.copyfrom_rev
e8f4e40f285a convert/subversion: work around memory leak in svn's python bindings
Bryan O'Sullivan <bos@serpentine.com>
parents: 4944
diff changeset
   170
        self.action = p.action
e8f4e40f285a convert/subversion: work around memory leak in svn's python bindings
Bryan O'Sullivan <bos@serpentine.com>
parents: 4944
diff changeset
   171
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   172
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   173
def get_log_child(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   174
    fp,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   175
    url,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   176
    paths,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   177
    start,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   178
    end,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   179
    limit=0,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   180
    discover_changed_paths=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   181
    strict_node_history=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   182
):
5127
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5122
diff changeset
   183
    protocol = -1
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   184
5127
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5122
diff changeset
   185
    def receiver(orig_paths, revnum, author, date, message, pool):
20057
d54467c1a198 convert: fix svn crash when svn.ra.get_log calls back with orig_paths=None
Mads Kiilerich <madski@unity3d.com>
parents: 19787
diff changeset
   186
        paths = {}
5127
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5122
diff changeset
   187
        if orig_paths is not None:
43105
649d3ac37a12 py3: define and use pycompat.iteritems() for hgext/
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
   188
            for k, v in pycompat.iteritems(orig_paths):
20057
d54467c1a198 convert: fix svn crash when svn.ra.get_log calls back with orig_paths=None
Mads Kiilerich <madski@unity3d.com>
parents: 19787
diff changeset
   189
                paths[k] = changedpath(v)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   190
        pickle.dump((paths, revnum, author, date, message), fp, protocol)
5143
d4fa6bafc43a Remove trailing spaces, fix indentation
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5141
diff changeset
   191
5127
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5122
diff changeset
   192
    try:
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5122
diff changeset
   193
        # Use an ra of our own so that our parent can consume
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5122
diff changeset
   194
        # our results without confusing the server.
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5122
diff changeset
   195
        t = transport.SvnRaTransport(url=url)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   196
        svn.ra.get_log(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   197
            t.ra,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   198
            paths,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   199
            start,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   200
            end,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   201
            limit,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   202
            discover_changed_paths,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   203
            strict_node_history,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   204
            receiver,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   205
        )
5873
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   206
    except IOError:
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   207
        # Caller may interrupt the iteration
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   208
        pickle.dump(None, fp, protocol)
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24856
diff changeset
   209
    except Exception as inst:
37087
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36843
diff changeset
   210
        pickle.dump(stringutil.forcebytestr(inst), fp, protocol)
5127
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5122
diff changeset
   211
    else:
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5122
diff changeset
   212
        pickle.dump(None, fp, protocol)
36541
d697e39f61a6 convert: avoid closing ui.fout in subversion code (issue5807)
Sascha Nemecek <nemecek@wienfluss.net>
parents: 36485
diff changeset
   213
    fp.flush()
6397
e1402cf5e08a convert: avoid svn log retrieval process cleanup
Patrick Mezard <pmezard@gmail.com>
parents: 6281
diff changeset
   214
    # With large history, cleanup process goes crazy and suddenly
e1402cf5e08a convert: avoid svn log retrieval process cleanup
Patrick Mezard <pmezard@gmail.com>
parents: 6281
diff changeset
   215
    # consumes *huge* amount of memory. The output file being closed,
e1402cf5e08a convert: avoid svn log retrieval process cleanup
Patrick Mezard <pmezard@gmail.com>
parents: 6281
diff changeset
   216
    # there is no need for clean termination.
e1402cf5e08a convert: avoid svn log retrieval process cleanup
Patrick Mezard <pmezard@gmail.com>
parents: 6281
diff changeset
   217
    os._exit(0)
5127
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5122
diff changeset
   218
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   219
5139
18abf13064cb Move debugsvnlog to subversion module.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5138
diff changeset
   220
def debugsvnlog(ui, **opts):
18abf13064cb Move debugsvnlog to subversion module.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5138
diff changeset
   221
    """Fetch SVN log in a subprocess and channel them back to parent to
18abf13064cb Move debugsvnlog to subversion module.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5138
diff changeset
   222
    avoid memory collection issues.
18abf13064cb Move debugsvnlog to subversion module.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5138
diff changeset
   223
    """
44827
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   224
    with util.with_lc_ctype():
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   225
        if svn is None:
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   226
            raise error.Abort(
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   227
                _(b'debugsvnlog could not load Subversion python bindings')
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   228
            )
17053
0e4171fdb9d4 convert: check for failed svn import in debugsvnlog and abort cleanly
Mads Kiilerich <mads@kiilerich.com>
parents: 17052
diff changeset
   229
44827
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   230
        args = decodeargs(ui.fin.read())
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   231
        get_log_child(ui.fout, *args)
5139
18abf13064cb Move debugsvnlog to subversion module.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5138
diff changeset
   232
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   233
8778
c5f36402daad use new style classes
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8764
diff changeset
   234
class logstream(object):
5873
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   235
    """Interruptible revision log iterator."""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   236
5873
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   237
    def __init__(self, stdout):
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   238
        self._stdout = stdout
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   239
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   240
    def __iter__(self):
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   241
        while True:
9587
d80a251baf9a convert/svn: better error when hg cannot call itself (issue1838)
Patrick Mezard <pmezard@gmail.com>
parents: 9521
diff changeset
   242
            try:
d80a251baf9a convert/svn: better error when hg cannot call itself (issue1838)
Patrick Mezard <pmezard@gmail.com>
parents: 9521
diff changeset
   243
                entry = pickle.load(self._stdout)
d80a251baf9a convert/svn: better error when hg cannot call itself (issue1838)
Patrick Mezard <pmezard@gmail.com>
parents: 9521
diff changeset
   244
            except EOFError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   245
                raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   246
                    _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   247
                        b'Mercurial failed to run itself, check'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   248
                        b' hg executable is in PATH'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   249
                    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   250
                )
5873
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   251
            try:
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   252
                orig_paths, revnum, author, date, message = entry
16688
cfb6682961b8 cleanup: replace naked excepts with more specific ones
Brodie Rao <brodie@sf.io>
parents: 16687
diff changeset
   253
            except (TypeError, ValueError):
5873
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   254
                if entry is None:
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   255
                    break
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   256
                raise error.Abort(_(b"log stream exception '%s'") % entry)
5873
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   257
            yield entry
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   258
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   259
    def close(self):
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   260
        if self._stdout:
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   261
            self._stdout.close()
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   262
            self._stdout = None
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
   263
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   264
20420
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
   265
class directlogstream(list):
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
   266
    """Direct revision log iterator.
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
   267
    This can be used for debugging and development but it will probably leak
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
   268
    memory and is not suitable for real conversions."""
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
   269
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   270
    def __init__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   271
        self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   272
        url,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   273
        paths,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   274
        start,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   275
        end,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   276
        limit=0,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   277
        discover_changed_paths=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   278
        strict_node_history=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   279
    ):
20420
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
   280
        def receiver(orig_paths, revnum, author, date, message, pool):
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
   281
            paths = {}
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
   282
            if orig_paths is not None:
43105
649d3ac37a12 py3: define and use pycompat.iteritems() for hgext/
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
   283
                for k, v in pycompat.iteritems(orig_paths):
20420
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
   284
                    paths[k] = changedpath(v)
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
   285
            self.append((paths, revnum, author, date, message))
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
   286
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
   287
        # Use an ra of our own so that our parent can consume
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
   288
        # our results without confusing the server.
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
   289
        t = transport.SvnRaTransport(url=url)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   290
        svn.ra.get_log(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   291
            t.ra,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   292
            paths,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   293
            start,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   294
            end,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   295
            limit,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   296
            discover_changed_paths,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   297
            strict_node_history,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   298
            receiver,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   299
        )
20420
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
   300
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
   301
    def close(self):
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
   302
        pass
8074
fc78313cba53 convert: Improved svn source detection.
Augie Fackler <durin42@gmail.com>
parents: 7973
diff changeset
   303
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   304
8074
fc78313cba53 convert: Improved svn source detection.
Augie Fackler <durin42@gmail.com>
parents: 7973
diff changeset
   305
# Check to see if the given path is a local Subversion repo. Verify this by
fc78313cba53 convert: Improved svn source detection.
Augie Fackler <durin42@gmail.com>
parents: 7973
diff changeset
   306
# looking for several svn-specific files and directories in the given
fc78313cba53 convert: Improved svn source detection.
Augie Fackler <durin42@gmail.com>
parents: 7973
diff changeset
   307
# directory.
9829
1b2516a547d4 convert/svn: delegate to svn bindings if HTTP probe fails
Patrick Mezard <pmezard@gmail.com>
parents: 9819
diff changeset
   308
def filecheck(ui, path, proto):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   309
    for x in (b'locks', b'hooks', b'format', b'db'):
8074
fc78313cba53 convert: Improved svn source detection.
Augie Fackler <durin42@gmail.com>
parents: 7973
diff changeset
   310
        if not os.path.exists(os.path.join(path, x)):
fc78313cba53 convert: Improved svn source detection.
Augie Fackler <durin42@gmail.com>
parents: 7973
diff changeset
   311
            return False
fc78313cba53 convert: Improved svn source detection.
Augie Fackler <durin42@gmail.com>
parents: 7973
diff changeset
   312
    return True
fc78313cba53 convert: Improved svn source detection.
Augie Fackler <durin42@gmail.com>
parents: 7973
diff changeset
   313
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   314
8074
fc78313cba53 convert: Improved svn source detection.
Augie Fackler <durin42@gmail.com>
parents: 7973
diff changeset
   315
# Check to see if a given path is the root of an svn repo over http. We verify
fc78313cba53 convert: Improved svn source detection.
Augie Fackler <durin42@gmail.com>
parents: 7973
diff changeset
   316
# this by requesting a version-controlled URL we know can't exist and looking
fc78313cba53 convert: Improved svn source detection.
Augie Fackler <durin42@gmail.com>
parents: 7973
diff changeset
   317
# for the svn-specific "not found" XML.
9829
1b2516a547d4 convert/svn: delegate to svn bindings if HTTP probe fails
Patrick Mezard <pmezard@gmail.com>
parents: 9819
diff changeset
   318
def httpcheck(ui, path, proto):
1b2516a547d4 convert/svn: delegate to svn bindings if HTTP probe fails
Patrick Mezard <pmezard@gmail.com>
parents: 9819
diff changeset
   319
    try:
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
   320
        opener = urlreq.buildopener()
44830
cb097496138a py3: pass URL as str
Manuel Jacob <me@manueljacob.de>
parents: 44829
diff changeset
   321
        rsp = opener.open(
cb097496138a py3: pass URL as str
Manuel Jacob <me@manueljacob.de>
parents: 44829
diff changeset
   322
            pycompat.strurl(b'%s://%s/!svn/ver/0/.svn' % (proto, path)), b'rb'
cb097496138a py3: pass URL as str
Manuel Jacob <me@manueljacob.de>
parents: 44829
diff changeset
   323
        )
10282
08a0f04b56bd many, many trivial check-code fixups
Matt Mackall <mpm@selenic.com>
parents: 10071
diff changeset
   324
        data = rsp.read()
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
   325
    except urlerr.httperror as inst:
9838
2e51cc30fc30 convert/svn: fix HTTP detection bug introduced by 1b2516a547d4
Patrick Mezard <pmezard@gmail.com>
parents: 9829
diff changeset
   326
        if inst.code != 404:
2e51cc30fc30 convert/svn: fix HTTP detection bug introduced by 1b2516a547d4
Patrick Mezard <pmezard@gmail.com>
parents: 9829
diff changeset
   327
            # Except for 404 we cannot know for sure this is not an svn repo
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   328
            ui.warn(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   329
                _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   330
                    b'svn: cannot probe remote repository, assume it could '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   331
                    b'be a subversion repository. Use --source-type if you '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   332
                    b'know better.\n'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   333
                )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   334
            )
9838
2e51cc30fc30 convert/svn: fix HTTP detection bug introduced by 1b2516a547d4
Patrick Mezard <pmezard@gmail.com>
parents: 9829
diff changeset
   335
            return True
2e51cc30fc30 convert/svn: fix HTTP detection bug introduced by 1b2516a547d4
Patrick Mezard <pmezard@gmail.com>
parents: 9829
diff changeset
   336
        data = inst.fp.read()
16689
f366d4c2ff34 cleanup: replace naked excepts with except Exception: ...
Brodie Rao <brodie@sf.io>
parents: 16688
diff changeset
   337
    except Exception:
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28861
diff changeset
   338
        # Could be urlerr.urlerror if the URL is invalid or anything else.
9829
1b2516a547d4 convert/svn: delegate to svn bindings if HTTP probe fails
Patrick Mezard <pmezard@gmail.com>
parents: 9819
diff changeset
   339
        return False
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   340
    return b'<m:human-readable errcode="160013">' in data
8074
fc78313cba53 convert: Improved svn source detection.
Augie Fackler <durin42@gmail.com>
parents: 7973
diff changeset
   341
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   342
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   343
protomap = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   344
    b'http': httpcheck,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   345
    b'https': httpcheck,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   346
    b'file': filecheck,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   347
}
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   348
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   349
9829
1b2516a547d4 convert/svn: delegate to svn bindings if HTTP probe fails
Patrick Mezard <pmezard@gmail.com>
parents: 9819
diff changeset
   350
def issvnurl(ui, url):
8764
46b5b4301fcc convert: default to file protocol when no :// found for svn repo url
Edouard Gomez <ed.gomez@free.fr>
parents: 8693
diff changeset
   351
    try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   352
        proto, path = url.split(b'://', 1)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   353
        if proto == b'file':
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   354
            if (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   355
                pycompat.iswindows
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   356
                and path[:1] == b'/'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   357
                and path[1:2].isalpha()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   358
                and path[2:6].lower() == b'%3a/'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   359
            ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   360
                path = path[:2] + b':/' + path[6:]
44814
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   361
            # pycompat.fsdecode() / pycompat.fsencode() are used so that bytes
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   362
            # in the URL roundtrip correctly on Unix. urlreq.url2pathname() on
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   363
            # py3 will decode percent-encoded bytes using the utf-8 encoding
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   364
            # and the "replace" error handler. This means that it will not
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   365
            # preserve non-UTF-8 bytes (https://bugs.python.org/issue40983).
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   366
            # url.open() uses the reverse function (urlreq.pathname2url()) and
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   367
            # has a similar problem
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   368
            # (https://bz.mercurial-scm.org/show_bug.cgi?id=6357). It makes
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   369
            # sense to solve both problems together and handle all file URLs
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   370
            # consistently. For now, we warn.
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   371
            unicodepath = urlreq.url2pathname(pycompat.fsdecode(path))
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   372
            if pycompat.ispy3 and u'\N{REPLACEMENT CHARACTER}' in unicodepath:
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   373
                ui.warn(
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   374
                    _(
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   375
                        b'on Python 3, we currently do not support non-UTF-8 '
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   376
                        b'percent-encoded bytes in file URLs for Subversion '
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   377
                        b'repositories\n'
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   378
                    )
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   379
                )
75b59d221aa3 py3: pass native string to urlreq.url2pathname()
Manuel Jacob <me@manueljacob.de>
parents: 44812
diff changeset
   380
            path = pycompat.fsencode(unicodepath)
8764
46b5b4301fcc convert: default to file protocol when no :// found for svn repo url
Edouard Gomez <ed.gomez@free.fr>
parents: 8693
diff changeset
   381
    except ValueError:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   382
        proto = b'file'
8764
46b5b4301fcc convert: default to file protocol when no :// found for svn repo url
Edouard Gomez <ed.gomez@free.fr>
parents: 8693
diff changeset
   383
        path = os.path.abspath(url)
44831
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
   384
        try:
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
   385
            path.decode(fsencoding)
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
   386
        except UnicodeDecodeError:
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
   387
            ui.warn(
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
   388
                _(
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
   389
                    b'Subversion requires that paths can be converted to '
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
   390
                    b'Unicode using the current locale encoding (%s)\n'
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
   391
                )
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
   392
                % pycompat.sysbytes(fsencoding)
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
   393
            )
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
   394
            return False
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   395
    if proto == b'file':
16067
467a85ced564 i18n: use util.pconvert() instead of 'str.replace()' for problematic encoding
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15970
diff changeset
   396
        path = util.pconvert(path)
44829
697212a830fb convert: bail out in Subversion source if encountering non-ASCII HTTP(S) URL
Manuel Jacob <me@manueljacob.de>
parents: 44827
diff changeset
   397
    elif proto in (b'http', 'https'):
697212a830fb convert: bail out in Subversion source if encountering non-ASCII HTTP(S) URL
Manuel Jacob <me@manueljacob.de>
parents: 44827
diff changeset
   398
        if not encoding.isasciistr(path):
697212a830fb convert: bail out in Subversion source if encountering non-ASCII HTTP(S) URL
Manuel Jacob <me@manueljacob.de>
parents: 44827
diff changeset
   399
            ui.warn(
697212a830fb convert: bail out in Subversion source if encountering non-ASCII HTTP(S) URL
Manuel Jacob <me@manueljacob.de>
parents: 44827
diff changeset
   400
                _(
697212a830fb convert: bail out in Subversion source if encountering non-ASCII HTTP(S) URL
Manuel Jacob <me@manueljacob.de>
parents: 44827
diff changeset
   401
                    b"Subversion sources don't support non-ASCII characters in "
697212a830fb convert: bail out in Subversion source if encountering non-ASCII HTTP(S) URL
Manuel Jacob <me@manueljacob.de>
parents: 44827
diff changeset
   402
                    b"HTTP(S) URLs. Please percent-encode them.\n"
697212a830fb convert: bail out in Subversion source if encountering non-ASCII HTTP(S) URL
Manuel Jacob <me@manueljacob.de>
parents: 44827
diff changeset
   403
                )
697212a830fb convert: bail out in Subversion source if encountering non-ASCII HTTP(S) URL
Manuel Jacob <me@manueljacob.de>
parents: 44827
diff changeset
   404
            )
697212a830fb convert: bail out in Subversion source if encountering non-ASCII HTTP(S) URL
Manuel Jacob <me@manueljacob.de>
parents: 44827
diff changeset
   405
            return False
10885
9e4d120e3c32 convert/subversion: fix default URL checker prototype
Patrick Mezard <pmezard@gmail.com>
parents: 10618
diff changeset
   406
    check = protomap.get(proto, lambda *args: False)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   407
    while b'/' in path:
9829
1b2516a547d4 convert/svn: delegate to svn bindings if HTTP probe fails
Patrick Mezard <pmezard@gmail.com>
parents: 9819
diff changeset
   408
        if check(ui, path, proto):
8074
fc78313cba53 convert: Improved svn source detection.
Augie Fackler <durin42@gmail.com>
parents: 7973
diff changeset
   409
            return True
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   410
        path = path.rsplit(b'/', 1)[0]
8074
fc78313cba53 convert: Improved svn source detection.
Augie Fackler <durin42@gmail.com>
parents: 7973
diff changeset
   411
    return False
fc78313cba53 convert: Improved svn source detection.
Augie Fackler <durin42@gmail.com>
parents: 7973
diff changeset
   412
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   413
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
   414
# SVN conversion code stolen from bzr-svn and tailor
5876
fb93c774dfff convert: document the subversion conversion model
Patrick Mezard <pmezard@gmail.com>
parents: 5875
diff changeset
   415
#
fb93c774dfff convert: document the subversion conversion model
Patrick Mezard <pmezard@gmail.com>
parents: 5875
diff changeset
   416
# Subversion looks like a versioned filesystem, branches structures
fb93c774dfff convert: document the subversion conversion model
Patrick Mezard <pmezard@gmail.com>
parents: 5875
diff changeset
   417
# are defined by conventions and not enforced by the tool. First,
fb93c774dfff convert: document the subversion conversion model
Patrick Mezard <pmezard@gmail.com>
parents: 5875
diff changeset
   418
# we define the potential branches (modules) as "trunk" and "branches"
fb93c774dfff convert: document the subversion conversion model
Patrick Mezard <pmezard@gmail.com>
parents: 5875
diff changeset
   419
# children directories. Revisions are then identified by their
fb93c774dfff convert: document the subversion conversion model
Patrick Mezard <pmezard@gmail.com>
parents: 5875
diff changeset
   420
# module and revision number (and a repository identifier).
fb93c774dfff convert: document the subversion conversion model
Patrick Mezard <pmezard@gmail.com>
parents: 5875
diff changeset
   421
#
fb93c774dfff convert: document the subversion conversion model
Patrick Mezard <pmezard@gmail.com>
parents: 5875
diff changeset
   422
# The revision graph is really a tree (or a forest). By default, a
fb93c774dfff convert: document the subversion conversion model
Patrick Mezard <pmezard@gmail.com>
parents: 5875
diff changeset
   423
# revision parent is the previous revision in the same module. If the
fb93c774dfff convert: document the subversion conversion model
Patrick Mezard <pmezard@gmail.com>
parents: 5875
diff changeset
   424
# module directory is copied/moved from another module then the
fb93c774dfff convert: document the subversion conversion model
Patrick Mezard <pmezard@gmail.com>
parents: 5875
diff changeset
   425
# revision is the module root and its parent the source revision in
fb93c774dfff convert: document the subversion conversion model
Patrick Mezard <pmezard@gmail.com>
parents: 5875
diff changeset
   426
# the parent module. A revision has at most one parent.
fb93c774dfff convert: document the subversion conversion model
Patrick Mezard <pmezard@gmail.com>
parents: 5875
diff changeset
   427
#
5438
6fa5258be3d4 convert: rename convert_svn to svn_source
Bryan O'Sullivan <bos@serpentine.com>
parents: 5437
diff changeset
   428
class svn_source(converter_source):
35187
671aba341d90 convert: save an indicator of the repo type for sources and sinks
Matt Harbison <matt_harbison@yahoo.com>
parents: 34890
diff changeset
   429
    def __init__(self, ui, repotype, url, revs=None):
671aba341d90 convert: save an indicator of the repo type for sources and sinks
Matt Harbison <matt_harbison@yahoo.com>
parents: 34890
diff changeset
   430
        super(svn_source, self).__init__(ui, repotype, url, revs=revs)
4807
15a3cbfc6568 convert: call superclass init from engine init functions
Brendan Cully <brendan@kublai.com>
parents: 4799
diff changeset
   431
44831
e3b19004087a convert: correctly convert paths to UTF-8 for Subversion
Manuel Jacob <me@manueljacob.de>
parents: 44830
diff changeset
   432
        init_fsencoding()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   433
        if not (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   434
            url.startswith(b'svn://')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   435
            or url.startswith(b'svn+ssh://')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   436
            or (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   437
                os.path.exists(url)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   438
                and os.path.exists(os.path.join(url, b'.svn'))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   439
            )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   440
            or issvnurl(ui, url)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   441
        ):
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   442
            raise NoRepo(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   443
                _(b"%s does not look like a Subversion repository") % url
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   444
            )
13480
69418d4525d1 convert/svn: abort operation when python bindings are not available
Azhagu Selvan SP <tamizhgeek@gmail.com>
parents: 13190
diff changeset
   445
        if svn is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   446
            raise MissingTool(_(b'could not load Subversion python bindings'))
7447
4fd92687f331 convert: improve reporting of invalid svn bindings
Patrick Mezard <pmezard@gmail.com>
parents: 7446
diff changeset
   447
4fd92687f331 convert: improve reporting of invalid svn bindings
Patrick Mezard <pmezard@gmail.com>
parents: 7446
diff changeset
   448
        try:
4fd92687f331 convert: improve reporting of invalid svn bindings
Patrick Mezard <pmezard@gmail.com>
parents: 7446
diff changeset
   449
            version = svn.core.SVN_VER_MAJOR, svn.core.SVN_VER_MINOR
4fd92687f331 convert: improve reporting of invalid svn bindings
Patrick Mezard <pmezard@gmail.com>
parents: 7446
diff changeset
   450
            if version < (1, 4):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   451
                raise MissingTool(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   452
                    _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   453
                        b'Subversion python bindings %d.%d found, '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   454
                        b'1.4 or later required'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   455
                    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   456
                    % version
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   457
                )
7447
4fd92687f331 convert: improve reporting of invalid svn bindings
Patrick Mezard <pmezard@gmail.com>
parents: 7446
diff changeset
   458
        except AttributeError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   459
            raise MissingTool(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   460
                _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   461
                    b'Subversion python bindings are too old, 1.4 '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   462
                    b'or later required'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   463
                )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   464
            )
4766
95cbb6b74790 convert: activate subversion engine
Brendan Cully <brendan@kublai.com>
parents: 4765
diff changeset
   465
4813
1fcdf2fe3d7c convert: svn: use revmap to parse only new revisions in incremental conversions
Brendan Cully <brendan@kublai.com>
parents: 4811
diff changeset
   466
        self.lastrevs = {}
1fcdf2fe3d7c convert: svn: use revmap to parse only new revisions in incremental conversions
Brendan Cully <brendan@kublai.com>
parents: 4811
diff changeset
   467
4766
95cbb6b74790 convert: activate subversion engine
Brendan Cully <brendan@kublai.com>
parents: 4765
diff changeset
   468
        latest = None
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
   469
        try:
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
   470
            # Support file://path@rev syntax. Useful e.g. to convert
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
   471
            # deleted branches.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   472
            at = url.rfind(b'@')
4927
5e89b0dafce5 convert/subversion.py: str.rsplit is not available in Python 2.3
Bryan O'Sullivan <bos@serpentine.com>
parents: 4925
diff changeset
   473
            if at >= 0:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   474
                latest = int(url[at + 1 :])
4927
5e89b0dafce5 convert/subversion.py: str.rsplit is not available in Python 2.3
Bryan O'Sullivan <bos@serpentine.com>
parents: 4925
diff changeset
   475
                url = url[:at]
7874
d812029cda85 cleanup: drop variables for unused return values
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7857
diff changeset
   476
        except ValueError:
4766
95cbb6b74790 convert: activate subversion engine
Brendan Cully <brendan@kublai.com>
parents: 4765
diff changeset
   477
            pass
5008
b6c3abdbe0eb convert: urlify svn repos if necessary.
Brendan Cully <brendan@kublai.com>
parents: 4960
diff changeset
   478
        self.url = geturl(url)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   479
        self.encoding = b'UTF-8'  # Subversion is always nominal UTF-8
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
   480
        try:
44827
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   481
            with util.with_lc_ctype():
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   482
                self.transport = transport.SvnRaTransport(url=self.url)
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   483
                self.ra = self.transport.ra
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   484
                self.ctx = self.transport.client
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   485
                self.baseurl = svn.ra.get_repos_root(self.ra)
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   486
                # Module is either empty or a repository path starting with
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   487
                # a slash and not ending with a slash.
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   488
                self.module = urlreq.unquote(self.url[len(self.baseurl) :])
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   489
                self.prevmodule = None
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   490
                self.rootmodule = self.module
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   491
                self.commits = {}
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   492
                self.paths = {}
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   493
                self.uuid = svn.ra.get_uuid(self.ra)
28460
d6dbb0968c13 convert: fix relative import of stdlib module in subversion
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 28459
diff changeset
   494
        except svn.core.SubversionException:
8206
cce63ef1045b ui: print_exc() -> traceback()
Matt Mackall <mpm@selenic.com>
parents: 8172
diff changeset
   495
            ui.traceback()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   496
            svnversion = b'%d.%d.%d' % (
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   497
                svn.core.SVN_VER_MAJOR,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   498
                svn.core.SVN_VER_MINOR,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   499
                svn.core.SVN_VER_MICRO,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   500
            )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   501
            raise NoRepo(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   502
                _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   503
                    b"%s does not look like a Subversion repository "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   504
                    b"to libsvn version %s"
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   505
                )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   506
                % (self.url, svnversion)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   507
            )
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
   508
25748
baea47cafe75 convert: add support for specifying multiple revs
Durham Goode <durham@fb.com>
parents: 25660
diff changeset
   509
        if revs:
baea47cafe75 convert: add support for specifying multiple revs
Durham Goode <durham@fb.com>
parents: 25660
diff changeset
   510
            if len(revs) > 1:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   511
                raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   512
                    _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   513
                        b'subversion source does not support '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   514
                        b'specifying multiple revisions'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   515
                    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   516
                )
5145
0751c25ea0c0 raise util.Abort again if specified revision is not an integer.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5144
diff changeset
   517
            try:
25748
baea47cafe75 convert: add support for specifying multiple revs
Durham Goode <durham@fb.com>
parents: 25660
diff changeset
   518
                latest = int(revs[0])
5145
0751c25ea0c0 raise util.Abort again if specified revision is not an integer.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5144
diff changeset
   519
            except ValueError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   520
                raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   521
                    _(b'svn: revision %s is not an integer') % revs[0]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   522
                )
5145
0751c25ea0c0 raise util.Abort again if specified revision is not an integer.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5144
diff changeset
   523
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   524
        trunkcfg = self.ui.config(b'convert', b'svn.trunk')
34890
effae88bccdb convert: register missed subversion config items
Augie Fackler <augie@google.com>
parents: 34645
diff changeset
   525
        if trunkcfg is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   526
            trunkcfg = b'trunk'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   527
        self.trunkname = trunkcfg.strip(b'/')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   528
        self.startrev = self.ui.config(b'convert', b'svn.startrev')
6173
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
   529
        try:
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
   530
            self.startrev = int(self.startrev)
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
   531
            if self.startrev < 0:
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
   532
                self.startrev = 0
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
   533
        except ValueError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   534
            raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   535
                _(b'svn: start revision %s is not an integer') % self.startrev
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   536
            )
6173
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
   537
14152
00121103546a convert: handle invalid subversion source paths
Mads Kiilerich <mads@kiilerich.com>
parents: 13970
diff changeset
   538
        try:
44827
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   539
            with util.with_lc_ctype():
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   540
                self.head = self.latest(self.module, latest)
14152
00121103546a convert: handle invalid subversion source paths
Mads Kiilerich <mads@kiilerich.com>
parents: 13970
diff changeset
   541
        except SvnPathNotFound:
00121103546a convert: handle invalid subversion source paths
Mads Kiilerich <mads@kiilerich.com>
parents: 13970
diff changeset
   542
            self.head = None
5957
971a17af5982 convert: prevent svn branches to leave the root module tree
Patrick Mezard <pmezard@gmail.com>
parents: 5956
diff changeset
   543
        if not self.head:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   544
            raise error.Abort(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   545
                _(b'no revision found in module %s') % self.module
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   546
            )
5955
c4496b7c10ce convert: fix svn_source.latest()
Patrick Mezard <pmezard@gmail.com>
parents: 5921
diff changeset
   547
        self.last_changed = self.revnum(self.head)
6210
942287cb1f57 Removed trailing spaces from everything except test output
Thomas Arendsen Hein <thomas@intevation.de>
parents: 6173
diff changeset
   548
22298
4ba35d4298a0 convert: refactor subversion getchanges and caching
Mads Kiilerich <madski@unity3d.com>
parents: 22297
diff changeset
   549
        self._changescache = (None, None)
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
   550
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   551
        if os.path.exists(os.path.join(url, b'.svn/entries')):
5554
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5537
diff changeset
   552
            self.wc = url
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5537
diff changeset
   553
        else:
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5537
diff changeset
   554
            self.wc = None
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5537
diff changeset
   555
        self.convertfp = None
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5537
diff changeset
   556
44827
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   557
    def before(self):
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   558
        self.with_lc_ctype = util.with_lc_ctype()
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   559
        self.with_lc_ctype.__enter__()
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   560
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   561
    def after(self):
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   562
        self.with_lc_ctype.__exit__(None, None, None)
5c0d5b48e58c convert: set LC_CTYPE around calls to Subversion bindings
Manuel Jacob <me@manueljacob.de>
parents: 44818
diff changeset
   563
5510
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5462
diff changeset
   564
    def setrevmap(self, revmap):
4840
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   565
        lastrevs = {}
36332
238646784294 py3: use default dict iterator instead of iterkeys
Augie Fackler <augie@google.com>
parents: 36186
diff changeset
   566
        for revid in revmap:
13690
af331f557942 convert/svn: extract revsplit() in a function
Patrick Mezard <pmezard@gmail.com>
parents: 13653
diff changeset
   567
            uuid, module, revnum = revsplit(revid)
4840
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   568
            lastrevnum = lastrevs.setdefault(module, revnum)
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   569
            if revnum > lastrevnum:
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   570
                lastrevs[module] = revnum
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   571
        self.lastrevs = lastrevs
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   572
4925
2642726b61b6 convert/subversion.py: fix bad assumptions about SVN path naming
Bryan O'Sullivan <bos@serpentine.com>
parents: 4873
diff changeset
   573
    def exists(self, path, optrev):
2642726b61b6 convert/subversion.py: fix bad assumptions about SVN path naming
Bryan O'Sullivan <bos@serpentine.com>
parents: 4873
diff changeset
   574
        try:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   575
            svn.client.ls(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   576
                self.url.rstrip(b'/') + b'/' + quote(path),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   577
                optrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   578
                False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   579
                self.ctx,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   580
            )
5461
ab4d2e9f3b97 convert: svn -- fix 'exists'
Kirill Smelkov <kirr@landau.phys.spbu.ru>
parents: 5438
diff changeset
   581
            return True
28460
d6dbb0968c13 convert: fix relative import of stdlib module in subversion
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 28459
diff changeset
   582
        except svn.core.SubversionException:
5461
ab4d2e9f3b97 convert: svn -- fix 'exists'
Kirill Smelkov <kirr@landau.phys.spbu.ru>
parents: 5438
diff changeset
   583
            return False
4925
2642726b61b6 convert/subversion.py: fix bad assumptions about SVN path naming
Bryan O'Sullivan <bos@serpentine.com>
parents: 4873
diff changeset
   584
4840
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   585
    def getheads(self):
6491
2bdd43246c95 convert: check svn branches are directories
Patrick Mezard <pmezard@gmail.com>
parents: 6400
diff changeset
   586
        def isdir(path, revnum):
6848
59c40c60d8d8 convert: remove leading slash from ra.check_path inputs (issue 1236)
Patrick Mezard <pmezard@gmail.com>
parents: 6847
diff changeset
   587
            kind = self._checkpath(path, revnum)
6491
2bdd43246c95 convert: check svn branches are directories
Patrick Mezard <pmezard@gmail.com>
parents: 6400
diff changeset
   588
            return kind == svn.core.svn_node_dir
2bdd43246c95 convert: check svn branches are directories
Patrick Mezard <pmezard@gmail.com>
parents: 6400
diff changeset
   589
5854
8b95f598097c convert: separate trunk detection from branch layout detection
Edouard Gomez <ed.gomez@free.fr>
parents: 5842
diff changeset
   590
        def getcfgpath(name, rev):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   591
            cfgpath = self.ui.config(b'convert', b'svn.' + name)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   592
            if cfgpath is not None and cfgpath.strip() == b'':
6172
0cd6846e5200 convert: allow svn trunk/branches/tags detection to be skipped
Patrick Mezard <pmezard@gmail.com>
parents: 6053
diff changeset
   593
                return None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   594
            path = (cfgpath or name).strip(b'/')
5854
8b95f598097c convert: separate trunk detection from branch layout detection
Edouard Gomez <ed.gomez@free.fr>
parents: 5842
diff changeset
   595
            if not self.exists(path, rev):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   596
                if self.module.endswith(path) and name == b'trunk':
13494
3178aca36b0f convert.svn: branch name which equals trunk means `default' branch (issue2653)
Pavel Boldin <boldin.pavel@gmail.com>
parents: 13480
diff changeset
   597
                    # we are converting from inside this directory
3178aca36b0f convert.svn: branch name which equals trunk means `default' branch (issue2653)
Pavel Boldin <boldin.pavel@gmail.com>
parents: 13480
diff changeset
   598
                    return None
5854
8b95f598097c convert: separate trunk detection from branch layout detection
Edouard Gomez <ed.gomez@free.fr>
parents: 5842
diff changeset
   599
                if cfgpath:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   600
                    raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   601
                        _(b'expected %s to be at %r, but not found')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   602
                        % (name, path)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   603
                    )
5854
8b95f598097c convert: separate trunk detection from branch layout detection
Edouard Gomez <ed.gomez@free.fr>
parents: 5842
diff changeset
   604
                return None
44816
7a4630536e53 py3: avoid using %r format on bytes
Manuel Jacob <me@manueljacob.de>
parents: 44815
diff changeset
   605
            self.ui.note(
7a4630536e53 py3: avoid using %r format on bytes
Manuel Jacob <me@manueljacob.de>
parents: 44815
diff changeset
   606
                _(b'found %s at %r\n') % (name, pycompat.bytestr(path))
7a4630536e53 py3: avoid using %r format on bytes
Manuel Jacob <me@manueljacob.de>
parents: 44815
diff changeset
   607
            )
5854
8b95f598097c convert: separate trunk detection from branch layout detection
Edouard Gomez <ed.gomez@free.fr>
parents: 5842
diff changeset
   608
            return path
8b95f598097c convert: separate trunk detection from branch layout detection
Edouard Gomez <ed.gomez@free.fr>
parents: 5842
diff changeset
   609
5117
c89ed1f27e78 convert: svn: add helper function for optrevs
Brendan Cully <brendan@kublai.com>
parents: 5114
diff changeset
   610
        rev = optrev(self.last_changed)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   611
        oldmodule = b''
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   612
        trunk = getcfgpath(b'trunk', rev)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   613
        self.tags = getcfgpath(b'tags', rev)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   614
        branches = getcfgpath(b'branches', rev)
5854
8b95f598097c convert: separate trunk detection from branch layout detection
Edouard Gomez <ed.gomez@free.fr>
parents: 5842
diff changeset
   615
8b95f598097c convert: separate trunk detection from branch layout detection
Edouard Gomez <ed.gomez@free.fr>
parents: 5842
diff changeset
   616
        # If the project has a trunk or branches, we will extract heads
8b95f598097c convert: separate trunk detection from branch layout detection
Edouard Gomez <ed.gomez@free.fr>
parents: 5842
diff changeset
   617
        # from them. We keep the project root otherwise.
8b95f598097c convert: separate trunk detection from branch layout detection
Edouard Gomez <ed.gomez@free.fr>
parents: 5842
diff changeset
   618
        if trunk:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   619
            oldmodule = self.module or b''
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   620
            self.module += b'/' + trunk
5955
c4496b7c10ce convert: fix svn_source.latest()
Patrick Mezard <pmezard@gmail.com>
parents: 5921
diff changeset
   621
            self.head = self.latest(self.module, self.last_changed)
5957
971a17af5982 convert: prevent svn branches to leave the root module tree
Patrick Mezard <pmezard@gmail.com>
parents: 5956
diff changeset
   622
            if not self.head:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   623
                raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   624
                    _(b'no revision found in module %s') % self.module
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   625
                )
5854
8b95f598097c convert: separate trunk detection from branch layout detection
Edouard Gomez <ed.gomez@free.fr>
parents: 5842
diff changeset
   626
8b95f598097c convert: separate trunk detection from branch layout detection
Edouard Gomez <ed.gomez@free.fr>
parents: 5842
diff changeset
   627
        # First head in the list is the module's head
8b95f598097c convert: separate trunk detection from branch layout detection
Edouard Gomez <ed.gomez@free.fr>
parents: 5842
diff changeset
   628
        self.heads = [self.head]
6400
c237b27e1350 convert: allow tags detection to be disabled
Patrick Mezard <pmezard@gmail.com>
parents: 6399
diff changeset
   629
        if self.tags is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   630
            self.tags = b'%s/%s' % (oldmodule, (self.tags or b'tags'))
5854
8b95f598097c convert: separate trunk detection from branch layout detection
Edouard Gomez <ed.gomez@free.fr>
parents: 5842
diff changeset
   631
8b95f598097c convert: separate trunk detection from branch layout detection
Edouard Gomez <ed.gomez@free.fr>
parents: 5842
diff changeset
   632
        # Check if branches bring a few more heads to the list
8b95f598097c convert: separate trunk detection from branch layout detection
Edouard Gomez <ed.gomez@free.fr>
parents: 5842
diff changeset
   633
        if branches:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   634
            rpath = self.url.strip(b'/')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   635
            branchnames = svn.client.ls(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   636
                rpath + b'/' + quote(branches), rev, False, self.ctx
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   637
            )
18374
942ecb55b1de convert: process subversion branch in a sorted order
Mads Kiilerich <mads@kiilerich.com>
parents: 17974
diff changeset
   638
            for branch in sorted(branchnames):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   639
                module = b'%s/%s/%s' % (oldmodule, branches, branch)
6491
2bdd43246c95 convert: check svn branches are directories
Patrick Mezard <pmezard@gmail.com>
parents: 6400
diff changeset
   640
                if not isdir(module, self.last_changed):
2bdd43246c95 convert: check svn branches are directories
Patrick Mezard <pmezard@gmail.com>
parents: 6400
diff changeset
   641
                    continue
5955
c4496b7c10ce convert: fix svn_source.latest()
Patrick Mezard <pmezard@gmail.com>
parents: 5921
diff changeset
   642
                brevid = self.latest(module, self.last_changed)
5957
971a17af5982 convert: prevent svn branches to leave the root module tree
Patrick Mezard <pmezard@gmail.com>
parents: 5956
diff changeset
   643
                if not brevid:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   644
                    self.ui.note(_(b'ignoring empty branch %s\n') % branch)
5957
971a17af5982 convert: prevent svn branches to leave the root module tree
Patrick Mezard <pmezard@gmail.com>
parents: 5956
diff changeset
   645
                    continue
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   646
                self.ui.note(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   647
                    _(b'found branch %s at %d\n')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   648
                    % (branch, self.revnum(brevid))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   649
                )
5955
c4496b7c10ce convert: fix svn_source.latest()
Patrick Mezard <pmezard@gmail.com>
parents: 5921
diff changeset
   650
                self.heads.append(brevid)
5462
91a522a69c13 convert: svn -- fix tags handling
Kirill Smelkov <kirr@landau.phys.spbu.ru>
parents: 5461
diff changeset
   651
6173
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
   652
        if self.startrev and self.heads:
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
   653
            if len(self.heads) > 1:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   654
                raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   655
                    _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   656
                        b'svn: start revision is not supported '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   657
                        b'with more than one branch'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   658
                    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   659
                )
6173
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
   660
            revnum = self.revnum(self.heads[0])
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
   661
            if revnum < self.startrev:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25748
diff changeset
   662
                raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   663
                    _(b'svn: no revision found after start revision %d')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   664
                    % self.startrev
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   665
                )
6173
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
   666
4840
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   667
        return self.heads
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   668
22300
35ab037de989 convert: introduce --full for converting all files
Mads Kiilerich <madski@unity3d.com>
parents: 22298
diff changeset
   669
    def _getchanges(self, rev, full):
5121
ef338e34a906 convert: look up copies in getchanges instead of getcommit
Brendan Cully <brendan@kublai.com>
parents: 5120
diff changeset
   670
        (paths, parents) = self.paths[rev]
22300
35ab037de989 convert: introduce --full for converting all files
Mads Kiilerich <madski@unity3d.com>
parents: 22298
diff changeset
   671
        copies = {}
5956
094638b3cbed convert: checkout svn root revisions
Patrick Mezard <pmezard@gmail.com>
parents: 5955
diff changeset
   672
        if parents:
11127
ab575114ddd8 convert/svn: do not retrieve removed files
Patrick Mezard <pmezard@gmail.com>
parents: 11126
diff changeset
   673
            files, self.removed, copies = self.expandpaths(rev, paths, parents)
22300
35ab037de989 convert: introduce --full for converting all files
Mads Kiilerich <madski@unity3d.com>
parents: 22298
diff changeset
   674
        if full or not parents:
5956
094638b3cbed convert: checkout svn root revisions
Patrick Mezard <pmezard@gmail.com>
parents: 5955
diff changeset
   675
            # Perform a full checkout on roots
13690
af331f557942 convert/svn: extract revsplit() in a function
Patrick Mezard <pmezard@gmail.com>
parents: 13653
diff changeset
   676
            uuid, module, revnum = revsplit(rev)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   677
            entries = svn.client.ls(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   678
                self.baseurl + quote(module), optrev(revnum), True, self.ctx
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   679
            )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   680
            files = [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   681
                n
43105
649d3ac37a12 py3: define and use pycompat.iteritems() for hgext/
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
   682
                for n, e in pycompat.iteritems(entries)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   683
                if e.kind == svn.core.svn_node_file
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   684
            ]
11127
ab575114ddd8 convert/svn: do not retrieve removed files
Patrick Mezard <pmezard@gmail.com>
parents: 11126
diff changeset
   685
            self.removed = set()
5956
094638b3cbed convert: checkout svn root revisions
Patrick Mezard <pmezard@gmail.com>
parents: 5955
diff changeset
   686
5121
ef338e34a906 convert: look up copies in getchanges instead of getcommit
Brendan Cully <brendan@kublai.com>
parents: 5120
diff changeset
   687
        files.sort()
44812
0c27d981131a py3: use `pycompat.ziplist()`
Manuel Jacob <me@manueljacob.de>
parents: 44811
diff changeset
   688
        files = pycompat.ziplist(files, [rev] * len(files))
22298
4ba35d4298a0 convert: refactor subversion getchanges and caching
Mads Kiilerich <madski@unity3d.com>
parents: 22297
diff changeset
   689
        return (files, copies)
5121
ef338e34a906 convert: look up copies in getchanges instead of getcommit
Brendan Cully <brendan@kublai.com>
parents: 5120
diff changeset
   690
22300
35ab037de989 convert: introduce --full for converting all files
Mads Kiilerich <madski@unity3d.com>
parents: 22298
diff changeset
   691
    def getchanges(self, rev, full):
22298
4ba35d4298a0 convert: refactor subversion getchanges and caching
Mads Kiilerich <madski@unity3d.com>
parents: 22297
diff changeset
   692
        # reuse cache from getchangedfiles
22300
35ab037de989 convert: introduce --full for converting all files
Mads Kiilerich <madski@unity3d.com>
parents: 22298
diff changeset
   693
        if self._changescache[0] == rev and not full:
22298
4ba35d4298a0 convert: refactor subversion getchanges and caching
Mads Kiilerich <madski@unity3d.com>
parents: 22297
diff changeset
   694
            (files, copies) = self._changescache[1]
4ba35d4298a0 convert: refactor subversion getchanges and caching
Mads Kiilerich <madski@unity3d.com>
parents: 22297
diff changeset
   695
        else:
22300
35ab037de989 convert: introduce --full for converting all files
Mads Kiilerich <madski@unity3d.com>
parents: 22298
diff changeset
   696
            (files, copies) = self._getchanges(rev, full)
22298
4ba35d4298a0 convert: refactor subversion getchanges and caching
Mads Kiilerich <madski@unity3d.com>
parents: 22297
diff changeset
   697
            # caller caches the result, so free it here to release memory
4ba35d4298a0 convert: refactor subversion getchanges and caching
Mads Kiilerich <madski@unity3d.com>
parents: 22297
diff changeset
   698
            del self.paths[rev]
24395
216fa1ba9993 convert: optimize convert of files that are unmodified from p2 in merges
Mads Kiilerich <madski@unity3d.com>
parents: 24306
diff changeset
   699
        return (files, copies, set())
4840
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   700
5382
d7d395fb7cd5 convert_svn: add --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5373
diff changeset
   701
    def getchangedfiles(self, rev, i):
22298
4ba35d4298a0 convert: refactor subversion getchanges and caching
Mads Kiilerich <madski@unity3d.com>
parents: 22297
diff changeset
   702
        # called from filemap - cache computed values for reuse in getchanges
22300
35ab037de989 convert: introduce --full for converting all files
Mads Kiilerich <madski@unity3d.com>
parents: 22298
diff changeset
   703
        (files, copies) = self._getchanges(rev, False)
22298
4ba35d4298a0 convert: refactor subversion getchanges and caching
Mads Kiilerich <madski@unity3d.com>
parents: 22297
diff changeset
   704
        self._changescache = (rev, (files, copies))
4ba35d4298a0 convert: refactor subversion getchanges and caching
Mads Kiilerich <madski@unity3d.com>
parents: 22297
diff changeset
   705
        return [f[0] for f in files]
5382
d7d395fb7cd5 convert_svn: add --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5373
diff changeset
   706
4840
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   707
    def getcommit(self, rev):
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   708
        if rev not in self.commits:
13690
af331f557942 convert/svn: extract revsplit() in a function
Patrick Mezard <pmezard@gmail.com>
parents: 13653
diff changeset
   709
            uuid, module, revnum = revsplit(rev)
4840
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   710
            self.module = module
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   711
            self.reparent(module)
5875
f1504d3355bb convert: fetch less revisions when looking for a branch parent
Patrick Mezard <pmezard@gmail.com>
parents: 5873
diff changeset
   712
            # We assume that:
f1504d3355bb convert: fetch less revisions when looking for a branch parent
Patrick Mezard <pmezard@gmail.com>
parents: 5873
diff changeset
   713
            # - requests for revisions after "stop" come from the
f1504d3355bb convert: fetch less revisions when looking for a branch parent
Patrick Mezard <pmezard@gmail.com>
parents: 5873
diff changeset
   714
            # revision graph backward traversal. Cache all of them
f1504d3355bb convert: fetch less revisions when looking for a branch parent
Patrick Mezard <pmezard@gmail.com>
parents: 5873
diff changeset
   715
            # down to stop, they will be used eventually.
f1504d3355bb convert: fetch less revisions when looking for a branch parent
Patrick Mezard <pmezard@gmail.com>
parents: 5873
diff changeset
   716
            # - requests for revisions before "stop" come to get
f1504d3355bb convert: fetch less revisions when looking for a branch parent
Patrick Mezard <pmezard@gmail.com>
parents: 5873
diff changeset
   717
            # isolated branches parents. Just fetch what is needed.
4840
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   718
            stop = self.lastrevs.get(module, 0)
5875
f1504d3355bb convert: fetch less revisions when looking for a branch parent
Patrick Mezard <pmezard@gmail.com>
parents: 5873
diff changeset
   719
            if revnum < stop:
f1504d3355bb convert: fetch less revisions when looking for a branch parent
Patrick Mezard <pmezard@gmail.com>
parents: 5873
diff changeset
   720
                stop = revnum + 1
5871
c1244685425a convert: fix parents of last fetched svn revision
Patrick Mezard <pmezard@gmail.com>
parents: 5870
diff changeset
   721
            self._fetch_revisions(revnum, stop)
15970
9f2ed48f8cda convert: subversion convert abort on revision not found (issue 3205)
Jesus Espino Garcia <jesus.espino@kaleidos.net>
parents: 15751
diff changeset
   722
            if rev not in self.commits:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   723
                raise error.Abort(_(b'svn: revision %s not found') % revnum)
22201
269688a398c4 cleanup: fix some list comprehension redefinitions of existing vars
Mads Kiilerich <madski@unity3d.com>
parents: 21635
diff changeset
   724
        revcommit = self.commits[rev]
4840
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   725
        # caller caches the result, so free it here to release memory
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   726
        del self.commits[rev]
22201
269688a398c4 cleanup: fix some list comprehension redefinitions of existing vars
Mads Kiilerich <madski@unity3d.com>
parents: 21635
diff changeset
   727
        return revcommit
4840
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   728
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   729
    def checkrevformat(self, revstr, mapname=b'splicemap'):
19122
83973dc1bfe9 splicemap: improve error handling when source is subversion (issue2084)
Ben Goswami <bengoswami@fb.com>
parents: 18374
diff changeset
   730
        """ fails if revision format does not match the correct format"""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   731
        if not re.match(
44817
423e20c78e6d py3: pass regex as bytes
Manuel Jacob <me@manueljacob.de>
parents: 44816
diff changeset
   732
            br'svn:[0-9a-f]{8,8}-[0-9a-f]{4,4}-'
423e20c78e6d py3: pass regex as bytes
Manuel Jacob <me@manueljacob.de>
parents: 44816
diff changeset
   733
            br'[0-9a-f]{4,4}-[0-9a-f]{4,4}-[0-9a-f]'
423e20c78e6d py3: pass regex as bytes
Manuel Jacob <me@manueljacob.de>
parents: 44816
diff changeset
   734
            br'{12,12}(.*)@[0-9]+$',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   735
            revstr,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   736
        ):
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   737
            raise error.Abort(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43105
diff changeset
   738
                _(b'%s entry %s is not a valid revision identifier')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   739
                % (mapname, revstr)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   740
            )
19122
83973dc1bfe9 splicemap: improve error handling when source is subversion (issue2084)
Ben Goswami <bengoswami@fb.com>
parents: 18374
diff changeset
   741
22414
299eaa09b41b convert: enable deterministic conversion progress bar for svn
Augie Fackler <raf@durin42.com>
parents: 22300
diff changeset
   742
    def numcommits(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   743
        return int(self.head.rsplit(b'@', 1)[1]) - self.startrev
22414
299eaa09b41b convert: enable deterministic conversion progress bar for svn
Augie Fackler <raf@durin42.com>
parents: 22300
diff changeset
   744
4840
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   745
    def gettags(self):
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   746
        tags = {}
6172
0cd6846e5200 convert: allow svn trunk/branches/tags detection to be skipped
Patrick Mezard <pmezard@gmail.com>
parents: 6053
diff changeset
   747
        if self.tags is None:
0cd6846e5200 convert: allow svn trunk/branches/tags detection to be skipped
Patrick Mezard <pmezard@gmail.com>
parents: 6053
diff changeset
   748
            return tags
6210
942287cb1f57 Removed trailing spaces from everything except test output
Thomas Arendsen Hein <thomas@intevation.de>
parents: 6173
diff changeset
   749
6399
5efd447a9b8d convert: follow svn tags history (issue953)
Patrick Mezard <pmezard@gmail.com>
parents: 6397
diff changeset
   750
        # svn tags are just a convention, project branches left in a
5efd447a9b8d convert: follow svn tags history (issue953)
Patrick Mezard <pmezard@gmail.com>
parents: 6397
diff changeset
   751
        # 'tags' directory. There is no other relationship than
5efd447a9b8d convert: follow svn tags history (issue953)
Patrick Mezard <pmezard@gmail.com>
parents: 6397
diff changeset
   752
        # ancestry, which is expensive to discover and makes them hard
5efd447a9b8d convert: follow svn tags history (issue953)
Patrick Mezard <pmezard@gmail.com>
parents: 6397
diff changeset
   753
        # to update incrementally.  Worse, past revisions may be
5efd447a9b8d convert: follow svn tags history (issue953)
Patrick Mezard <pmezard@gmail.com>
parents: 6397
diff changeset
   754
        # referenced by tags far away in the future, requiring a deep
5efd447a9b8d convert: follow svn tags history (issue953)
Patrick Mezard <pmezard@gmail.com>
parents: 6397
diff changeset
   755
        # history traversal on every calculation.  Current code
5efd447a9b8d convert: follow svn tags history (issue953)
Patrick Mezard <pmezard@gmail.com>
parents: 6397
diff changeset
   756
        # performs a single backward traversal, tracking moves within
5efd447a9b8d convert: follow svn tags history (issue953)
Patrick Mezard <pmezard@gmail.com>
parents: 6397
diff changeset
   757
        # the tags directory (tag renaming) and recording a new tag
5efd447a9b8d convert: follow svn tags history (issue953)
Patrick Mezard <pmezard@gmail.com>
parents: 6397
diff changeset
   758
        # everytime a project is copied from outside the tags
5efd447a9b8d convert: follow svn tags history (issue953)
Patrick Mezard <pmezard@gmail.com>
parents: 6397
diff changeset
   759
        # directory. It also lists deleted tags, this behaviour may
5efd447a9b8d convert: follow svn tags history (issue953)
Patrick Mezard <pmezard@gmail.com>
parents: 6397
diff changeset
   760
        # change in the future.
5efd447a9b8d convert: follow svn tags history (issue953)
Patrick Mezard <pmezard@gmail.com>
parents: 6397
diff changeset
   761
        pendings = []
5efd447a9b8d convert: follow svn tags history (issue953)
Patrick Mezard <pmezard@gmail.com>
parents: 6397
diff changeset
   762
        tagspath = self.tags
5efd447a9b8d convert: follow svn tags history (issue953)
Patrick Mezard <pmezard@gmail.com>
parents: 6397
diff changeset
   763
        start = svn.ra.get_latest_revnum(self.ra)
11195
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   764
        stream = self._getlog([self.tags], start, self.startrev)
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   765
        try:
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   766
            for entry in stream:
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   767
                origpaths, revnum, author, date, message = entry
19468
82c512643bed convert: catch empty origpaths in svn gettags (issue3941)
Matt Mackall <mpm@selenic.com>
parents: 19122
diff changeset
   768
                if not origpaths:
82c512643bed convert: catch empty origpaths in svn gettags (issue3941)
Matt Mackall <mpm@selenic.com>
parents: 19122
diff changeset
   769
                    origpaths = []
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   770
                copies = [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   771
                    (e.copyfrom_path, e.copyfrom_rev, p)
43105
649d3ac37a12 py3: define and use pycompat.iteritems() for hgext/
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
   772
                    for p, e in pycompat.iteritems(origpaths)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   773
                    if e.copyfrom_path
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   774
                ]
11195
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   775
                # Apply moves/copies from more specific to general
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   776
                copies.sort(reverse=True)
6399
5efd447a9b8d convert: follow svn tags history (issue953)
Patrick Mezard <pmezard@gmail.com>
parents: 6397
diff changeset
   777
11195
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   778
                srctagspath = tagspath
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   779
                if copies and copies[-1][2] == tagspath:
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   780
                    # Track tags directory moves
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   781
                    srctagspath = copies.pop()[0]
6399
5efd447a9b8d convert: follow svn tags history (issue953)
Patrick Mezard <pmezard@gmail.com>
parents: 6397
diff changeset
   782
11195
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   783
                for source, sourcerev, dest in copies:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   784
                    if not dest.startswith(tagspath + b'/'):
11195
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   785
                        continue
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   786
                    for tag in pendings:
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   787
                        if tag[0].startswith(dest):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   788
                            tagpath = source + tag[0][len(dest) :]
11195
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   789
                            tag[:2] = [tagpath, sourcerev]
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   790
                            break
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   791
                    else:
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   792
                        pendings.append([source, sourcerev, dest])
8248
d093e57671ea convert/svn: ignore composite tags
Patrick Mezard <pmezard@gmail.com>
parents: 8247
diff changeset
   793
11195
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   794
                # Filter out tags with children coming from different
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   795
                # parts of the repository like:
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   796
                # /tags/tag.1 (from /trunk:10)
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   797
                # /tags/tag.1/foo (from /branches/foo:12)
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   798
                # Here/tags/tag.1 discarded as well as its children.
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   799
                # It happens with tools like cvs2svn. Such tags cannot
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   800
                # be represented in mercurial.
44470
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 44024
diff changeset
   801
                addeds = {
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 44024
diff changeset
   802
                    p: e.copyfrom_path
43105
649d3ac37a12 py3: define and use pycompat.iteritems() for hgext/
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
   803
                    for p, e in pycompat.iteritems(origpaths)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   804
                    if e.action == b'A' and e.copyfrom_path
44470
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 44024
diff changeset
   805
                }
11195
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   806
                badroots = set()
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   807
                for destroot in addeds:
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   808
                    for source, sourcerev, dest in pendings:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   809
                        if not dest.startswith(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   810
                            destroot + b'/'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   811
                        ) or source.startswith(addeds[destroot] + b'/'):
11195
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   812
                            continue
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   813
                        badroots.add(destroot)
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   814
                        break
8248
d093e57671ea convert/svn: ignore composite tags
Patrick Mezard <pmezard@gmail.com>
parents: 8247
diff changeset
   815
11195
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   816
                for badroot in badroots:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   817
                    pendings = [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   818
                        p
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   819
                        for p in pendings
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   820
                        if p[2] != badroot
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   821
                        and not p[2].startswith(badroot + b'/')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   822
                    ]
6399
5efd447a9b8d convert: follow svn tags history (issue953)
Patrick Mezard <pmezard@gmail.com>
parents: 6397
diff changeset
   823
11195
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   824
                # Tell tag renamings from tag creations
15124
8542a9c9f679 convert: rename local variable
Martin Geisler <mg@aragost.com>
parents: 14273
diff changeset
   825
                renamings = []
11195
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   826
                for source, sourcerev, dest in pendings:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   827
                    tagname = dest.split(b'/')[-1]
11195
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   828
                    if source.startswith(srctagspath):
15124
8542a9c9f679 convert: rename local variable
Martin Geisler <mg@aragost.com>
parents: 14273
diff changeset
   829
                        renamings.append([source, sourcerev, tagname])
11195
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   830
                        continue
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   831
                    if tagname in tags:
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   832
                        # Keep the latest tag value
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   833
                        continue
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   834
                    # From revision may be fake, get one with changes
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   835
                    try:
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   836
                        tagid = self.latest(source, sourcerev)
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   837
                        if tagid and tagname not in tags:
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   838
                            tags[tagname] = tagid
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   839
                    except SvnPathNotFound:
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   840
                        # It happens when we are following directories
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   841
                        # we assumed were copied with their parents
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   842
                        # but were really created in the tag
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   843
                        # directory.
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   844
                        pass
15124
8542a9c9f679 convert: rename local variable
Martin Geisler <mg@aragost.com>
parents: 14273
diff changeset
   845
                pendings = renamings
11195
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   846
                tagspath = srctagspath
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   847
        finally:
46bb49134498 convert/svn: close gettags() log stream (issue2196)
Aaron Digulla <digulla@hepe.com>
parents: 11194
diff changeset
   848
            stream.close()
4946
e8f4e40f285a convert/subversion: work around memory leak in svn's python bindings
Bryan O'Sullivan <bos@serpentine.com>
parents: 4944
diff changeset
   849
        return tags
4840
a265fe42abe7 convert: svn code movement (no actual changes)
Brendan Cully <brendan@kublai.com>
parents: 4839
diff changeset
   850
5554
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5537
diff changeset
   851
    def converted(self, rev, destrev):
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5537
diff changeset
   852
        if not self.wc:
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5537
diff changeset
   853
            return
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5537
diff changeset
   854
        if self.convertfp is None:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   855
            self.convertfp = open(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   856
                os.path.join(self.wc, b'.svn', b'hg-shamap'), b'ab'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   857
            )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   858
        self.convertfp.write(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   859
            util.tonativeeol(b'%s %d\n' % (destrev, self.revnum(rev)))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   860
        )
5554
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5537
diff changeset
   861
        self.convertfp.flush()
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5537
diff changeset
   862
4810
c2d529f288a1 convert: move some code into common init function
Brendan Cully <brendan@kublai.com>
parents: 4807
diff changeset
   863
    def revid(self, revnum, module=None):
44811
8439351d3208 py3: use `%d` for int in % formatting
Manuel Jacob <me@manueljacob.de>
parents: 44809
diff changeset
   864
        return b'svn:%s%s@%d' % (self.uuid, module or self.module, revnum)
4774
d305852a5ec8 convert: svn: add revnum() to convert rev to revnum
Brendan Cully <brendan@kublai.com>
parents: 4773
diff changeset
   865
d305852a5ec8 convert: svn: add revnum() to convert rev to revnum
Brendan Cully <brendan@kublai.com>
parents: 4773
diff changeset
   866
    def revnum(self, rev):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   867
        return int(rev.split(b'@')[-1])
4789
c5dd8e184279 convert: svn: add function to get the latest revision touching a path
Brendan Cully <brendan@kublai.com>
parents: 4788
diff changeset
   868
16464
0e1329d905df convert/svn: clarify svn_source.latest() stop arg default value
Patrick Mezard <patrick@mezard.eu>
parents: 16162
diff changeset
   869
    def latest(self, path, stop=None):
0e1329d905df convert/svn: clarify svn_source.latest() stop arg default value
Patrick Mezard <patrick@mezard.eu>
parents: 16162
diff changeset
   870
        """Find the latest revid affecting path, up to stop revision
0e1329d905df convert/svn: clarify svn_source.latest() stop arg default value
Patrick Mezard <patrick@mezard.eu>
parents: 16162
diff changeset
   871
        number. If stop is None, default to repository latest
0e1329d905df convert/svn: clarify svn_source.latest() stop arg default value
Patrick Mezard <patrick@mezard.eu>
parents: 16162
diff changeset
   872
        revision. It may return a revision in a different module,
0e1329d905df convert/svn: clarify svn_source.latest() stop arg default value
Patrick Mezard <patrick@mezard.eu>
parents: 16162
diff changeset
   873
        since a branch may be moved without a change being
0e1329d905df convert/svn: clarify svn_source.latest() stop arg default value
Patrick Mezard <patrick@mezard.eu>
parents: 16162
diff changeset
   874
        reported. Return None if computed module does not belong to
0e1329d905df convert/svn: clarify svn_source.latest() stop arg default value
Patrick Mezard <patrick@mezard.eu>
parents: 16162
diff changeset
   875
        rootmodule subtree.
5955
c4496b7c10ce convert: fix svn_source.latest()
Patrick Mezard <pmezard@gmail.com>
parents: 5921
diff changeset
   876
        """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   877
16466
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   878
        def findchanges(path, start, stop=None):
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   879
            stream = self._getlog([path], start, stop or 1)
16465
ad38b96c88f9 convert/svn: refactor svn_source.latest() with a nested function
Patrick Mezard <patrick@mezard.eu>
parents: 16464
diff changeset
   880
            try:
ad38b96c88f9 convert/svn: refactor svn_source.latest() with a nested function
Patrick Mezard <patrick@mezard.eu>
parents: 16464
diff changeset
   881
                for entry in stream:
ad38b96c88f9 convert/svn: refactor svn_source.latest() with a nested function
Patrick Mezard <patrick@mezard.eu>
parents: 16464
diff changeset
   882
                    paths, revnum, author, date, message = entry
16466
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   883
                    if stop is None and paths:
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   884
                        # We do not know the latest changed revision,
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   885
                        # keep the first one with changed paths.
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   886
                        break
44818
9e5f598fd29b py3: fix comparison between int and None
Manuel Jacob <me@manueljacob.de>
parents: 44817
diff changeset
   887
                    if stop is not None and revnum <= stop:
16465
ad38b96c88f9 convert/svn: refactor svn_source.latest() with a nested function
Patrick Mezard <patrick@mezard.eu>
parents: 16464
diff changeset
   888
                        break
ad38b96c88f9 convert/svn: refactor svn_source.latest() with a nested function
Patrick Mezard <patrick@mezard.eu>
parents: 16464
diff changeset
   889
ad38b96c88f9 convert/svn: refactor svn_source.latest() with a nested function
Patrick Mezard <patrick@mezard.eu>
parents: 16464
diff changeset
   890
                    for p in paths:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   891
                        if not path.startswith(p) or not paths[p].copyfrom_path:
16465
ad38b96c88f9 convert/svn: refactor svn_source.latest() with a nested function
Patrick Mezard <patrick@mezard.eu>
parents: 16464
diff changeset
   892
                            continue
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   893
                        newpath = paths[p].copyfrom_path + path[len(p) :]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   894
                        self.ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   895
                            b"branch renamed from %s to %s at %d\n"
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   896
                            % (path, newpath, revnum)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   897
                        )
16465
ad38b96c88f9 convert/svn: refactor svn_source.latest() with a nested function
Patrick Mezard <patrick@mezard.eu>
parents: 16464
diff changeset
   898
                        path = newpath
ad38b96c88f9 convert/svn: refactor svn_source.latest() with a nested function
Patrick Mezard <patrick@mezard.eu>
parents: 16464
diff changeset
   899
                        break
16466
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   900
                if not paths:
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   901
                    revnum = None
16465
ad38b96c88f9 convert/svn: refactor svn_source.latest() with a nested function
Patrick Mezard <patrick@mezard.eu>
parents: 16464
diff changeset
   902
                return revnum, path
ad38b96c88f9 convert/svn: refactor svn_source.latest() with a nested function
Patrick Mezard <patrick@mezard.eu>
parents: 16464
diff changeset
   903
            finally:
ad38b96c88f9 convert/svn: refactor svn_source.latest() with a nested function
Patrick Mezard <patrick@mezard.eu>
parents: 16464
diff changeset
   904
                stream.close()
ad38b96c88f9 convert/svn: refactor svn_source.latest() with a nested function
Patrick Mezard <patrick@mezard.eu>
parents: 16464
diff changeset
   905
6281
3e5f99b13998 convert: avoid querying log of foreign svn branches
Patrick Mezard <pmezard@gmail.com>
parents: 6214
diff changeset
   906
        if not path.startswith(self.rootmodule):
3e5f99b13998 convert: avoid querying log of foreign svn branches
Patrick Mezard <pmezard@gmail.com>
parents: 6214
diff changeset
   907
            # Requests on foreign branches may be forbidden at server level
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   908
            self.ui.debug(b'ignoring foreign branch %r\n' % path)
6281
3e5f99b13998 convert: avoid querying log of foreign svn branches
Patrick Mezard <pmezard@gmail.com>
parents: 6214
diff changeset
   909
            return None
3e5f99b13998 convert: avoid querying log of foreign svn branches
Patrick Mezard <pmezard@gmail.com>
parents: 6214
diff changeset
   910
16464
0e1329d905df convert/svn: clarify svn_source.latest() stop arg default value
Patrick Mezard <patrick@mezard.eu>
parents: 16162
diff changeset
   911
        if stop is None:
4789
c5dd8e184279 convert: svn: add function to get the latest revision touching a path
Brendan Cully <brendan@kublai.com>
parents: 4788
diff changeset
   912
            stop = svn.ra.get_latest_revnum(self.ra)
c5dd8e184279 convert: svn: add function to get the latest revision touching a path
Brendan Cully <brendan@kublai.com>
parents: 4788
diff changeset
   913
        try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   914
            prevmodule = self.reparent(b'')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   915
            dirent = svn.ra.stat(self.ra, path.strip(b'/'), stop)
6847
9d6d1fdb7d72 convert: restore previous svn transport parent correctly
Patrick Mezard <pmezard@gmail.com>
parents: 6553
diff changeset
   916
            self.reparent(prevmodule)
28460
d6dbb0968c13 convert: fix relative import of stdlib module in subversion
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 28459
diff changeset
   917
        except svn.core.SubversionException:
4789
c5dd8e184279 convert: svn: add function to get the latest revision touching a path
Brendan Cully <brendan@kublai.com>
parents: 4788
diff changeset
   918
            dirent = None
c5dd8e184279 convert: svn: add function to get the latest revision touching a path
Brendan Cully <brendan@kublai.com>
parents: 4788
diff changeset
   919
        if not dirent:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   920
            raise SvnPathNotFound(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   921
                _(b'%s not found up to revision %d') % (path, stop)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   922
            )
4789
c5dd8e184279 convert: svn: add function to get the latest revision touching a path
Brendan Cully <brendan@kublai.com>
parents: 4788
diff changeset
   923
8660
b9ec04eb2aa3 convert/subversion: wrap long lines in comments
Martin Geisler <mg@lazybytes.net>
parents: 8339
diff changeset
   924
        # stat() gives us the previous revision on this line of
b9ec04eb2aa3 convert/subversion: wrap long lines in comments
Martin Geisler <mg@lazybytes.net>
parents: 8339
diff changeset
   925
        # development, but it might be in *another module*. Fetch the
b9ec04eb2aa3 convert/subversion: wrap long lines in comments
Martin Geisler <mg@lazybytes.net>
parents: 8339
diff changeset
   926
        # log and detect renames down to the latest revision.
16465
ad38b96c88f9 convert/svn: refactor svn_source.latest() with a nested function
Patrick Mezard <patrick@mezard.eu>
parents: 16464
diff changeset
   927
        revnum, realpath = findchanges(path, stop, dirent.created_rev)
16466
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   928
        if revnum is None:
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   929
            # Tools like svnsync can create empty revision, when
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   930
            # synchronizing only a subtree for instance. These empty
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   931
            # revisions created_rev still have their original values
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   932
            # despite all changes having disappeared and can be
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   933
            # returned by ra.stat(), at least when stating the root
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   934
            # module. In that case, do not trust created_rev and scan
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   935
            # the whole history.
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   936
            revnum, realpath = findchanges(path, stop)
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   937
            if revnum is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   938
                self.ui.debug(b'ignoring empty branch %r\n' % realpath)
16466
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   939
                return None
c53a49c345e1 convert/svn: do not try converting empty head revisions (issue3347)
Patrick Mezard <patrick@mezard.eu>
parents: 16465
diff changeset
   940
16465
ad38b96c88f9 convert/svn: refactor svn_source.latest() with a nested function
Patrick Mezard <patrick@mezard.eu>
parents: 16464
diff changeset
   941
        if not realpath.startswith(self.rootmodule):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   942
            self.ui.debug(b'ignoring foreign branch %r\n' % realpath)
5957
971a17af5982 convert: prevent svn branches to leave the root module tree
Patrick Mezard <pmezard@gmail.com>
parents: 5956
diff changeset
   943
            return None
16465
ad38b96c88f9 convert/svn: refactor svn_source.latest() with a nested function
Patrick Mezard <patrick@mezard.eu>
parents: 16464
diff changeset
   944
        return self.revid(revnum, realpath)
4789
c5dd8e184279 convert: svn: add function to get the latest revision touching a path
Brendan Cully <brendan@kublai.com>
parents: 4788
diff changeset
   945
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
   946
    def reparent(self, module):
6847
9d6d1fdb7d72 convert: restore previous svn transport parent correctly
Patrick Mezard <pmezard@gmail.com>
parents: 6553
diff changeset
   947
        """Reparent the svn transport and return the previous parent."""
9d6d1fdb7d72 convert: restore previous svn transport parent correctly
Patrick Mezard <pmezard@gmail.com>
parents: 6553
diff changeset
   948
        if self.prevmodule == module:
9d6d1fdb7d72 convert: restore previous svn transport parent correctly
Patrick Mezard <pmezard@gmail.com>
parents: 6553
diff changeset
   949
            return module
15599
c6be93a4c378 convert/svn: fix URL quoting issue with svn 1.7
Patrick Mezard <pmezard@gmail.com>
parents: 15381
diff changeset
   950
        svnurl = self.baseurl + quote(module)
6847
9d6d1fdb7d72 convert: restore previous svn transport parent correctly
Patrick Mezard <pmezard@gmail.com>
parents: 6553
diff changeset
   951
        prevmodule = self.prevmodule
9d6d1fdb7d72 convert: restore previous svn transport parent correctly
Patrick Mezard <pmezard@gmail.com>
parents: 6553
diff changeset
   952
        if prevmodule is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   953
            prevmodule = b''
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   954
        self.ui.debug(b"reparent to %s\n" % svnurl)
7074
b1a4f67b98d0 convert: properly encode subversion URLs (issue 1224)
Patrick Mezard <pmezard@gmail.com>
parents: 6877
diff changeset
   955
        svn.ra.reparent(self.ra, svnurl)
6847
9d6d1fdb7d72 convert: restore previous svn transport parent correctly
Patrick Mezard <pmezard@gmail.com>
parents: 6553
diff changeset
   956
        self.prevmodule = module
9d6d1fdb7d72 convert: restore previous svn transport parent correctly
Patrick Mezard <pmezard@gmail.com>
parents: 6553
diff changeset
   957
        return prevmodule
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
   958
5120
514c06098e9c convert: svn: pull up path to file expansion code into separate function.
Brendan Cully <brendan@kublai.com>
parents: 5119
diff changeset
   959
    def expandpaths(self, rev, paths, parents):
11127
ab575114ddd8 convert/svn: do not retrieve removed files
Patrick Mezard <pmezard@gmail.com>
parents: 11126
diff changeset
   960
        changed, removed = set(), set()
5120
514c06098e9c convert: svn: pull up path to file expansion code into separate function.
Brendan Cully <brendan@kublai.com>
parents: 5119
diff changeset
   961
        copies = {}
514c06098e9c convert: svn: pull up path to file expansion code into separate function.
Brendan Cully <brendan@kublai.com>
parents: 5119
diff changeset
   962
13690
af331f557942 convert/svn: extract revsplit() in a function
Patrick Mezard <pmezard@gmail.com>
parents: 13653
diff changeset
   963
        new_module, revnum = revsplit(rev)[1:]
5872
7d4149cccc5d convert: fix cross-branches subversion revisions handling
Patrick Mezard <pmezard@gmail.com>
parents: 5871
diff changeset
   964
        if new_module != self.module:
7d4149cccc5d convert: fix cross-branches subversion revisions handling
Patrick Mezard <pmezard@gmail.com>
parents: 5871
diff changeset
   965
            self.module = new_module
7d4149cccc5d convert: fix cross-branches subversion revisions handling
Patrick Mezard <pmezard@gmail.com>
parents: 5871
diff changeset
   966
            self.reparent(self.module)
5121
ef338e34a906 convert: look up copies in getchanges instead of getcommit
Brendan Cully <brendan@kublai.com>
parents: 5120
diff changeset
   967
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   968
        progress = self.ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   969
            _(b'scanning paths'), unit=_(b'paths'), total=len(paths)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   970
        )
11137
94afe7bc87a9 convert/svn: report path discovery progress
Patrick Mezard <pmezard@gmail.com>
parents: 11134
diff changeset
   971
        for i, (path, ent) in enumerate(paths):
38412
65d1d7da63d1 convert: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
   972
            progress.update(i, item=path)
6539
7814d7bb77bc convert: rename get_entry_from_path() into an svn_source method
Patrick Mezard <pmezard@gmail.com>
parents: 6538
diff changeset
   973
            entrypath = self.getrelpath(path)
5120
514c06098e9c convert: svn: pull up path to file expansion code into separate function.
Brendan Cully <brendan@kublai.com>
parents: 5119
diff changeset
   974
6848
59c40c60d8d8 convert: remove leading slash from ra.check_path inputs (issue 1236)
Patrick Mezard <pmezard@gmail.com>
parents: 6847
diff changeset
   975
            kind = self._checkpath(entrypath, revnum)
5120
514c06098e9c convert: svn: pull up path to file expansion code into separate function.
Brendan Cully <brendan@kublai.com>
parents: 5119
diff changeset
   976
            if kind == svn.core.svn_node_file:
11127
ab575114ddd8 convert/svn: do not retrieve removed files
Patrick Mezard <pmezard@gmail.com>
parents: 11126
diff changeset
   977
                changed.add(self.recode(entrypath))
6546
075b2c9aed37 convert: fix svn file copy detection code
Patrick Mezard <pmezard@gmail.com>
parents: 6545
diff changeset
   978
                if not ent.copyfrom_path or not parents:
6544
3447c088a516 convert: cleanup svn file copy handling
Patrick Mezard <pmezard@gmail.com>
parents: 6543
diff changeset
   979
                    continue
8660
b9ec04eb2aa3 convert/subversion: wrap long lines in comments
Martin Geisler <mg@lazybytes.net>
parents: 8339
diff changeset
   980
                # Copy sources not in parent revisions cannot be
b9ec04eb2aa3 convert/subversion: wrap long lines in comments
Martin Geisler <mg@lazybytes.net>
parents: 8339
diff changeset
   981
                # represented, ignore their origin for now
13690
af331f557942 convert/svn: extract revsplit() in a function
Patrick Mezard <pmezard@gmail.com>
parents: 13653
diff changeset
   982
                pmodule, prevnum = revsplit(parents[0])[1:]
6546
075b2c9aed37 convert: fix svn file copy detection code
Patrick Mezard <pmezard@gmail.com>
parents: 6545
diff changeset
   983
                if ent.copyfrom_rev < prevnum:
075b2c9aed37 convert: fix svn file copy detection code
Patrick Mezard <pmezard@gmail.com>
parents: 6545
diff changeset
   984
                    continue
075b2c9aed37 convert: fix svn file copy detection code
Patrick Mezard <pmezard@gmail.com>
parents: 6545
diff changeset
   985
                copyfrom_path = self.getrelpath(ent.copyfrom_path, pmodule)
6544
3447c088a516 convert: cleanup svn file copy handling
Patrick Mezard <pmezard@gmail.com>
parents: 6543
diff changeset
   986
                if not copyfrom_path:
3447c088a516 convert: cleanup svn file copy handling
Patrick Mezard <pmezard@gmail.com>
parents: 6543
diff changeset
   987
                    continue
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   988
                self.ui.debug(
44815
d545b895234a py3: use `%d` for int in % formatting
Manuel Jacob <me@manueljacob.de>
parents: 44814
diff changeset
   989
                    b"copied to %s from %s@%d\n"
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   990
                    % (entrypath, copyfrom_path, ent.copyfrom_rev)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   991
                )
8885
2c184bd2b37a convert/svn: remove confusing unicode variable
Patrick Mezard <pmezard@gmail.com>
parents: 8884
diff changeset
   992
                copies[self.recode(entrypath)] = self.recode(copyfrom_path)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
   993
            elif kind == 0:  # gone, but had better be a deleted *file*
44815
d545b895234a py3: use `%d` for int in % formatting
Manuel Jacob <me@manueljacob.de>
parents: 44814
diff changeset
   994
                self.ui.debug(b"gone from %d\n" % ent.copyfrom_rev)
13690
af331f557942 convert/svn: extract revsplit() in a function
Patrick Mezard <pmezard@gmail.com>
parents: 13653
diff changeset
   995
                pmodule, prevnum = revsplit(parents[0])[1:]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   996
                parentpath = pmodule + b"/" + entrypath
11128
a9b427b5821e convert/svn: handle files/links replaced by dirs (issue2166)
Patrick Mezard <pmezard@gmail.com>
parents: 11127
diff changeset
   997
                fromkind = self._checkpath(entrypath, prevnum, pmodule)
6210
942287cb1f57 Removed trailing spaces from everything except test output
Thomas Arendsen Hein <thomas@intevation.de>
parents: 6173
diff changeset
   998
8881
76d3cd914c5e convert/svn: remove dead code and obsolete comments
Patrick Mezard <pmezard@gmail.com>
parents: 8843
diff changeset
   999
                if fromkind == svn.core.svn_node_file:
11127
ab575114ddd8 convert/svn: do not retrieve removed files
Patrick Mezard <pmezard@gmail.com>
parents: 11126
diff changeset
  1000
                    removed.add(self.recode(entrypath))
5120
514c06098e9c convert: svn: pull up path to file expansion code into separate function.
Brendan Cully <brendan@kublai.com>
parents: 5119
diff changeset
  1001
                elif fromkind == svn.core.svn_node_dir:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1002
                    oroot = parentpath.strip(b'/')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1003
                    nroot = path.strip(b'/')
11133
d7b6aab612b4 convert/svn: remove useless sort
Patrick Mezard <pmezard@gmail.com>
parents: 11132
diff changeset
  1004
                    children = self._iterfiles(oroot, prevnum)
11132
321b8b2aa20f convert/svn: list files explicitely, stop checking their type
Patrick Mezard <pmezard@gmail.com>
parents: 11130
diff changeset
  1005
                    for childpath in children:
321b8b2aa20f convert/svn: list files explicitely, stop checking their type
Patrick Mezard <pmezard@gmail.com>
parents: 11130
diff changeset
  1006
                        childpath = childpath.replace(oroot, nroot)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1007
                        childpath = self.getrelpath(b"/" + childpath, pmodule)
11125
7a5a98fc38e9 convert/svn: remove broken but unused copy filtering code
Patrick Mezard <pmezard@gmail.com>
parents: 11123
diff changeset
  1008
                        if childpath:
11127
ab575114ddd8 convert/svn: do not retrieve removed files
Patrick Mezard <pmezard@gmail.com>
parents: 11126
diff changeset
  1009
                            removed.add(self.recode(childpath))
5120
514c06098e9c convert: svn: pull up path to file expansion code into separate function.
Brendan Cully <brendan@kublai.com>
parents: 5119
diff changeset
  1010
                else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1011
                    self.ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1012
                        b'unknown path in revision %d: %s\n' % (revnum, path)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1013
                    )
12770
614f0d8724ab check-code: find trailing whitespace
Martin Geisler <mg@lazybytes.net>
parents: 12343
diff changeset
  1014
            elif kind == svn.core.svn_node_dir:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1015
                if ent.action == b'M':
11128
a9b427b5821e convert/svn: handle files/links replaced by dirs (issue2166)
Patrick Mezard <pmezard@gmail.com>
parents: 11127
diff changeset
  1016
                    # If the directory just had a prop change,
a9b427b5821e convert/svn: handle files/links replaced by dirs (issue2166)
Patrick Mezard <pmezard@gmail.com>
parents: 11127
diff changeset
  1017
                    # then we shouldn't need to look for its children.
5870
0d583ade5817 convert: don't scan directories on property changes
Patrick Mezard <pmezard@gmail.com>
parents: 5854
diff changeset
  1018
                    continue
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1019
                if ent.action == b'R' and parents:
11128
a9b427b5821e convert/svn: handle files/links replaced by dirs (issue2166)
Patrick Mezard <pmezard@gmail.com>
parents: 11127
diff changeset
  1020
                    # If a directory is replacing a file, mark the previous
a9b427b5821e convert/svn: handle files/links replaced by dirs (issue2166)
Patrick Mezard <pmezard@gmail.com>
parents: 11127
diff changeset
  1021
                    # file as deleted
13690
af331f557942 convert/svn: extract revsplit() in a function
Patrick Mezard <pmezard@gmail.com>
parents: 13653
diff changeset
  1022
                    pmodule, prevnum = revsplit(parents[0])[1:]
11128
a9b427b5821e convert/svn: handle files/links replaced by dirs (issue2166)
Patrick Mezard <pmezard@gmail.com>
parents: 11127
diff changeset
  1023
                    pkind = self._checkpath(entrypath, prevnum, pmodule)
a9b427b5821e convert/svn: handle files/links replaced by dirs (issue2166)
Patrick Mezard <pmezard@gmail.com>
parents: 11127
diff changeset
  1024
                    if pkind == svn.core.svn_node_file:
a9b427b5821e convert/svn: handle files/links replaced by dirs (issue2166)
Patrick Mezard <pmezard@gmail.com>
parents: 11127
diff changeset
  1025
                        removed.add(self.recode(entrypath))
13052
5fb924ee44d5 convert/svn: fix changed files list upon directory replacements
Patrick Mezard <pmezard@gmail.com>
parents: 12770
diff changeset
  1026
                    elif pkind == svn.core.svn_node_dir:
5fb924ee44d5 convert/svn: fix changed files list upon directory replacements
Patrick Mezard <pmezard@gmail.com>
parents: 12770
diff changeset
  1027
                        # We do not know what files were kept or removed,
5fb924ee44d5 convert/svn: fix changed files list upon directory replacements
Patrick Mezard <pmezard@gmail.com>
parents: 12770
diff changeset
  1028
                        # mark them all as changed.
5fb924ee44d5 convert/svn: fix changed files list upon directory replacements
Patrick Mezard <pmezard@gmail.com>
parents: 12770
diff changeset
  1029
                        for childpath in self._iterfiles(pmodule, prevnum):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1030
                            childpath = self.getrelpath(b"/" + childpath)
13052
5fb924ee44d5 convert/svn: fix changed files list upon directory replacements
Patrick Mezard <pmezard@gmail.com>
parents: 12770
diff changeset
  1031
                            if childpath:
5fb924ee44d5 convert/svn: fix changed files list upon directory replacements
Patrick Mezard <pmezard@gmail.com>
parents: 12770
diff changeset
  1032
                                changed.add(self.recode(childpath))
5870
0d583ade5817 convert: don't scan directories on property changes
Patrick Mezard <pmezard@gmail.com>
parents: 5854
diff changeset
  1033
11133
d7b6aab612b4 convert/svn: remove useless sort
Patrick Mezard <pmezard@gmail.com>
parents: 11132
diff changeset
  1034
                for childpath in self._iterfiles(path, revnum):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1035
                    childpath = self.getrelpath(b"/" + childpath)
11132
321b8b2aa20f convert/svn: list files explicitely, stop checking their type
Patrick Mezard <pmezard@gmail.com>
parents: 11130
diff changeset
  1036
                    if childpath:
321b8b2aa20f convert/svn: list files explicitely, stop checking their type
Patrick Mezard <pmezard@gmail.com>
parents: 11130
diff changeset
  1037
                        changed.add(self.recode(childpath))
5120
514c06098e9c convert: svn: pull up path to file expansion code into separate function.
Brendan Cully <brendan@kublai.com>
parents: 5119
diff changeset
  1038
8881
76d3cd914c5e convert/svn: remove dead code and obsolete comments
Patrick Mezard <pmezard@gmail.com>
parents: 8843
diff changeset
  1039
                # Handle directory copies
6543
a6e2e60b34d0 convert: handle past or foreign partial svn copies
Patrick Mezard <pmezard@gmail.com>
parents: 6542
diff changeset
  1040
                if not ent.copyfrom_path or not parents:
6542
e7810e61f7c1 convert: more cleanup in svn directory copy handling
Patrick Mezard <pmezard@gmail.com>
parents: 6541
diff changeset
  1041
                    continue
8660
b9ec04eb2aa3 convert/subversion: wrap long lines in comments
Martin Geisler <mg@lazybytes.net>
parents: 8339
diff changeset
  1042
                # Copy sources not in parent revisions cannot be
b9ec04eb2aa3 convert/subversion: wrap long lines in comments
Martin Geisler <mg@lazybytes.net>
parents: 8339
diff changeset
  1043
                # represented, ignore their origin for now
13690
af331f557942 convert/svn: extract revsplit() in a function
Patrick Mezard <pmezard@gmail.com>
parents: 13653
diff changeset
  1044
                pmodule, prevnum = revsplit(parents[0])[1:]
6543
a6e2e60b34d0 convert: handle past or foreign partial svn copies
Patrick Mezard <pmezard@gmail.com>
parents: 6542
diff changeset
  1045
                if ent.copyfrom_rev < prevnum:
a6e2e60b34d0 convert: handle past or foreign partial svn copies
Patrick Mezard <pmezard@gmail.com>
parents: 6542
diff changeset
  1046
                    continue
8882
48a04961b0dd convert/svn: remove useless encoding/decoding calls (issue1676)
Patrick Mezard <pmezard@gmail.com>
parents: 8881
diff changeset
  1047
                copyfrompath = self.getrelpath(ent.copyfrom_path, pmodule)
6542
e7810e61f7c1 convert: more cleanup in svn directory copy handling
Patrick Mezard <pmezard@gmail.com>
parents: 6541
diff changeset
  1048
                if not copyfrompath:
e7810e61f7c1 convert: more cleanup in svn directory copy handling
Patrick Mezard <pmezard@gmail.com>
parents: 6541
diff changeset
  1049
                    continue
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1050
                self.ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1051
                    b"mark %s came from %s:%d\n"
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1052
                    % (path, copyfrompath, ent.copyfrom_rev)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1053
                )
11133
d7b6aab612b4 convert/svn: remove useless sort
Patrick Mezard <pmezard@gmail.com>
parents: 11132
diff changeset
  1054
                children = self._iterfiles(ent.copyfrom_path, ent.copyfrom_rev)
11132
321b8b2aa20f convert/svn: list files explicitely, stop checking their type
Patrick Mezard <pmezard@gmail.com>
parents: 11130
diff changeset
  1055
                for childpath in children:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1056
                    childpath = self.getrelpath(b"/" + childpath, pmodule)
11132
321b8b2aa20f convert/svn: list files explicitely, stop checking their type
Patrick Mezard <pmezard@gmail.com>
parents: 11130
diff changeset
  1057
                    if not childpath:
6542
e7810e61f7c1 convert: more cleanup in svn directory copy handling
Patrick Mezard <pmezard@gmail.com>
parents: 6541
diff changeset
  1058
                        continue
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1059
                    copytopath = path + childpath[len(copyfrompath) :]
6542
e7810e61f7c1 convert: more cleanup in svn directory copy handling
Patrick Mezard <pmezard@gmail.com>
parents: 6541
diff changeset
  1060
                    copytopath = self.getrelpath(copytopath)
11132
321b8b2aa20f convert/svn: list files explicitely, stop checking their type
Patrick Mezard <pmezard@gmail.com>
parents: 11130
diff changeset
  1061
                    copies[self.recode(copytopath)] = self.recode(childpath)
5120
514c06098e9c convert: svn: pull up path to file expansion code into separate function.
Brendan Cully <brendan@kublai.com>
parents: 5119
diff changeset
  1062
38412
65d1d7da63d1 convert: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 38197
diff changeset
  1063
        progress.complete()
11127
ab575114ddd8 convert/svn: do not retrieve removed files
Patrick Mezard <pmezard@gmail.com>
parents: 11126
diff changeset
  1064
        changed.update(removed)
ab575114ddd8 convert/svn: do not retrieve removed files
Patrick Mezard <pmezard@gmail.com>
parents: 11126
diff changeset
  1065
        return (list(changed), removed, copies)
5120
514c06098e9c convert: svn: pull up path to file expansion code into separate function.
Brendan Cully <brendan@kublai.com>
parents: 5119
diff changeset
  1066
5871
c1244685425a convert: fix parents of last fetched svn revision
Patrick Mezard <pmezard@gmail.com>
parents: 5870
diff changeset
  1067
    def _fetch_revisions(self, from_revnum, to_revnum):
c1244685425a convert: fix parents of last fetched svn revision
Patrick Mezard <pmezard@gmail.com>
parents: 5870
diff changeset
  1068
        if from_revnum < to_revnum:
c1244685425a convert: fix parents of last fetched svn revision
Patrick Mezard <pmezard@gmail.com>
parents: 5870
diff changeset
  1069
            from_revnum, to_revnum = to_revnum, from_revnum
c1244685425a convert: fix parents of last fetched svn revision
Patrick Mezard <pmezard@gmail.com>
parents: 5870
diff changeset
  1070
4940
961379b2c586 convert/subversion: reduce memory usage by filtering early
Bryan O'Sullivan <bos@serpentine.com>
parents: 4927
diff changeset
  1071
        self.child_cset = None
6545
a7492fb2107b convert: fix svn branch source detection corner case
Patrick Mezard <pmezard@gmail.com>
parents: 6544
diff changeset
  1072
4946
e8f4e40f285a convert/subversion: work around memory leak in svn's python bindings
Bryan O'Sullivan <bos@serpentine.com>
parents: 4944
diff changeset
  1073
        def parselogentry(orig_paths, revnum, author, date, message):
6210
942287cb1f57 Removed trailing spaces from everything except test output
Thomas Arendsen Hein <thomas@intevation.de>
parents: 6173
diff changeset
  1074
            """Return the parsed commit object or None, and True if
5872
7d4149cccc5d convert: fix cross-branches subversion revisions handling
Patrick Mezard <pmezard@gmail.com>
parents: 5871
diff changeset
  1075
            the revision is a branch root.
7d4149cccc5d convert: fix cross-branches subversion revisions handling
Patrick Mezard <pmezard@gmail.com>
parents: 5871
diff changeset
  1076
            """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1077
            self.ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1078
                b"parsing revision %d (%d changes)\n"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1079
                % (revnum, len(orig_paths))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1080
            )
4940
961379b2c586 convert/subversion: reduce memory usage by filtering early
Bryan O'Sullivan <bos@serpentine.com>
parents: 4927
diff changeset
  1081
5957
971a17af5982 convert: prevent svn branches to leave the root module tree
Patrick Mezard <pmezard@gmail.com>
parents: 5956
diff changeset
  1082
            branched = False
4810
c2d529f288a1 convert: move some code into common init function
Brendan Cully <brendan@kublai.com>
parents: 4807
diff changeset
  1083
            rev = self.revid(revnum)
4837
4cdbaa885d8a convert: svn: some improvements in memory usage
Brendan Cully <brendan@kublai.com>
parents: 4814
diff changeset
  1084
            # branch log might return entries for a parent we already have
5871
c1244685425a convert: fix parents of last fetched svn revision
Patrick Mezard <pmezard@gmail.com>
parents: 5870
diff changeset
  1085
8117
2b30d8488819 remove unnecessary outer parenthesis in if-statements
Martin Geisler <mg@lazybytes.net>
parents: 8086
diff changeset
  1086
            if rev in self.commits or revnum < to_revnum:
5957
971a17af5982 convert: prevent svn branches to leave the root module tree
Patrick Mezard <pmezard@gmail.com>
parents: 5956
diff changeset
  1087
                return None, branched
4837
4cdbaa885d8a convert: svn: some improvements in memory usage
Brendan Cully <brendan@kublai.com>
parents: 4814
diff changeset
  1088
5120
514c06098e9c convert: svn: pull up path to file expansion code into separate function.
Brendan Cully <brendan@kublai.com>
parents: 5119
diff changeset
  1089
            parents = []
5958
59dce24933ad convert: follow svn module parent moves
Patrick Mezard <pmezard@gmail.com>
parents: 5957
diff changeset
  1090
            # check whether this revision is the start of a branch or part
59dce24933ad convert: follow svn module parent moves
Patrick Mezard <pmezard@gmail.com>
parents: 5957
diff changeset
  1091
            # of a branch renaming
43105
649d3ac37a12 py3: define and use pycompat.iteritems() for hgext/
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
  1092
            orig_paths = sorted(pycompat.iteritems(orig_paths))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1093
            root_paths = [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1094
                (p, e) for p, e in orig_paths if self.module.startswith(p)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1095
            ]
5958
59dce24933ad convert: follow svn module parent moves
Patrick Mezard <pmezard@gmail.com>
parents: 5957
diff changeset
  1096
            if root_paths:
59dce24933ad convert: follow svn module parent moves
Patrick Mezard <pmezard@gmail.com>
parents: 5957
diff changeset
  1097
                path, ent = root_paths[-1]
5119
4cef0b93969d convert: svn: hoist up branch creation check
Brendan Cully <brendan@kublai.com>
parents: 5118
diff changeset
  1098
                if ent.copyfrom_path:
5957
971a17af5982 convert: prevent svn branches to leave the root module tree
Patrick Mezard <pmezard@gmail.com>
parents: 5956
diff changeset
  1099
                    branched = True
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1100
                    newpath = ent.copyfrom_path + self.module[len(path) :]
5119
4cef0b93969d convert: svn: hoist up branch creation check
Brendan Cully <brendan@kublai.com>
parents: 5118
diff changeset
  1101
                    # ent.copyfrom_rev may not be the actual last revision
7476
6644c111f9e9 convert: backout a7492fb2107b
Patrick Mezard <pmezard@gmail.com>
parents: 7447
diff changeset
  1102
                    previd = self.latest(newpath, ent.copyfrom_rev)
5957
971a17af5982 convert: prevent svn branches to leave the root module tree
Patrick Mezard <pmezard@gmail.com>
parents: 5956
diff changeset
  1103
                    if previd is not None:
13690
af331f557942 convert/svn: extract revsplit() in a function
Patrick Mezard <pmezard@gmail.com>
parents: 13653
diff changeset
  1104
                        prevmodule, prevnum = revsplit(previd)[1:]
6173
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
  1105
                        if prevnum >= self.startrev:
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
  1106
                            parents = [previd]
10282
08a0f04b56bd many, many trivial check-code fixups
Matt Mackall <mpm@selenic.com>
parents: 10071
diff changeset
  1107
                            self.ui.note(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1108
                                _(b'found parent of branch %s at %d: %s\n')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1109
                                % (self.module, prevnum, prevmodule)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1110
                            )
5119
4cef0b93969d convert: svn: hoist up branch creation check
Brendan Cully <brendan@kublai.com>
parents: 5118
diff changeset
  1111
                else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1112
                    self.ui.debug(b"no copyfrom path, don't know what to do.\n")
5119
4cef0b93969d convert: svn: hoist up branch creation check
Brendan Cully <brendan@kublai.com>
parents: 5118
diff changeset
  1113
5120
514c06098e9c convert: svn: pull up path to file expansion code into separate function.
Brendan Cully <brendan@kublai.com>
parents: 5119
diff changeset
  1114
            paths = []
514c06098e9c convert: svn: pull up path to file expansion code into separate function.
Brendan Cully <brendan@kublai.com>
parents: 5119
diff changeset
  1115
            # filter out unrelated paths
4940
961379b2c586 convert/subversion: reduce memory usage by filtering early
Bryan O'Sullivan <bos@serpentine.com>
parents: 4927
diff changeset
  1116
            for path, ent in orig_paths:
6540
55bd855fc0af convert: improve subversion branch filtering
Patrick Mezard <pmezard@gmail.com>
parents: 6539
diff changeset
  1117
                if self.getrelpath(path) is None:
4788
62e1b6412b62 convert: svn: add an early return to move most changeset parsing out an indent level
Brendan Cully <brendan@kublai.com>
parents: 4787
diff changeset
  1118
                    continue
5120
514c06098e9c convert: svn: pull up path to file expansion code into separate function.
Brendan Cully <brendan@kublai.com>
parents: 5119
diff changeset
  1119
                paths.append((path, ent))
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1120
4788
62e1b6412b62 convert: svn: add an early return to move most changeset parsing out an indent level
Brendan Cully <brendan@kublai.com>
parents: 4787
diff changeset
  1121
            # Example SVN datetime. Includes microseconds.
62e1b6412b62 convert: svn: add an early return to move most changeset parsing out an indent level
Brendan Cully <brendan@kublai.com>
parents: 4787
diff changeset
  1122
            # ISO-8601 conformant
62e1b6412b62 convert: svn: add an early return to move most changeset parsing out an indent level
Brendan Cully <brendan@kublai.com>
parents: 4787
diff changeset
  1123
            # '2007-01-04T17:35:00.902377Z'
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1124
            date = dateutil.parsedate(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1125
                date[:19] + b" UTC", [b"%Y-%m-%dT%H:%M:%S"]
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1126
            )
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1127
            if self.ui.configbool(b'convert', b'localtimezone'):
17974
337d728e644f convert: add config option to use the local time zone
Julian Cowley <julian@lava.net>
parents: 17537
diff changeset
  1128
                date = makedatetimestamp(date[0])
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1129
24306
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 23583
diff changeset
  1130
            if message:
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 23583
diff changeset
  1131
                log = self.recode(message)
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 23583
diff changeset
  1132
            else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1133
                log = b''
24306
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 23583
diff changeset
  1134
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 23583
diff changeset
  1135
            if author:
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 23583
diff changeset
  1136
                author = self.recode(author)
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 23583
diff changeset
  1137
            else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1138
                author = b''
24306
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 23583
diff changeset
  1139
5120
514c06098e9c convert: svn: pull up path to file expansion code into separate function.
Brendan Cully <brendan@kublai.com>
parents: 5119
diff changeset
  1140
            try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1141
                branch = self.module.split(b"/")[-1]
13529
9b62cbe81f44 convert/svn: read trunk name once, use None for default
Patrick Mezard <pmezard@gmail.com>
parents: 13494
diff changeset
  1142
                if branch == self.trunkname:
9b62cbe81f44 convert/svn: read trunk name once, use None for default
Patrick Mezard <pmezard@gmail.com>
parents: 13494
diff changeset
  1143
                    branch = None
5120
514c06098e9c convert: svn: pull up path to file expansion code into separate function.
Brendan Cully <brendan@kublai.com>
parents: 5119
diff changeset
  1144
            except IndexError:
514c06098e9c convert: svn: pull up path to file expansion code into separate function.
Brendan Cully <brendan@kublai.com>
parents: 5119
diff changeset
  1145
                branch = None
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1146
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1147
            cset = commit(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1148
                author=author,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1149
                date=dateutil.datestr(date, b'%Y-%m-%d %H:%M:%S %1%2'),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1150
                desc=log,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1151
                parents=parents,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1152
                branch=branch,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1153
                rev=rev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1154
            )
4788
62e1b6412b62 convert: svn: add an early return to move most changeset parsing out an indent level
Brendan Cully <brendan@kublai.com>
parents: 4787
diff changeset
  1155
4796
26857a6f9dd0 convert: svn: pull out broken batching code, add alpha tags support
Brendan Cully <brendan@kublai.com>
parents: 4795
diff changeset
  1156
            self.commits[rev] = cset
5872
7d4149cccc5d convert: fix cross-branches subversion revisions handling
Patrick Mezard <pmezard@gmail.com>
parents: 5871
diff changeset
  1157
            # The parents list is *shared* among self.paths and the
7d4149cccc5d convert: fix cross-branches subversion revisions handling
Patrick Mezard <pmezard@gmail.com>
parents: 5871
diff changeset
  1158
            # commit object. Both will be updated below.
7d4149cccc5d convert: fix cross-branches subversion revisions handling
Patrick Mezard <pmezard@gmail.com>
parents: 5871
diff changeset
  1159
            self.paths[rev] = (paths, cset.parents)
4796
26857a6f9dd0 convert: svn: pull out broken batching code, add alpha tags support
Brendan Cully <brendan@kublai.com>
parents: 4795
diff changeset
  1160
            if self.child_cset and not self.child_cset.parents:
5872
7d4149cccc5d convert: fix cross-branches subversion revisions handling
Patrick Mezard <pmezard@gmail.com>
parents: 5871
diff changeset
  1161
                self.child_cset.parents[:] = [rev]
4788
62e1b6412b62 convert: svn: add an early return to move most changeset parsing out an indent level
Brendan Cully <brendan@kublai.com>
parents: 4787
diff changeset
  1162
            self.child_cset = cset
5957
971a17af5982 convert: prevent svn branches to leave the root module tree
Patrick Mezard <pmezard@gmail.com>
parents: 5956
diff changeset
  1163
            return cset, branched
4796
26857a6f9dd0 convert: svn: pull out broken batching code, add alpha tags support
Brendan Cully <brendan@kublai.com>
parents: 4795
diff changeset
  1164
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1165
        self.ui.note(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1166
            _(b'fetching revision log for "%s" from %d to %d\n')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1167
            % (self.module, from_revnum, to_revnum)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1168
        )
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1169
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1170
        try:
5871
c1244685425a convert: fix parents of last fetched svn revision
Patrick Mezard <pmezard@gmail.com>
parents: 5870
diff changeset
  1171
            firstcset = None
6173
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
  1172
            lastonbranch = False
6850
c6bb8fae3bd2 convert: normalize paths sent to svn get_log (issue 1219)
Patrick Mezard <pmezard@gmail.com>
parents: 6848
diff changeset
  1173
            stream = self._getlog([self.module], from_revnum, to_revnum)
5873
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
  1174
            try:
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
  1175
                for entry in stream:
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
  1176
                    paths, revnum, author, date, message = entry
6173
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
  1177
                    if revnum < self.startrev:
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
  1178
                        lastonbranch = True
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
  1179
                        break
8172
36b935cad495 Fix subversion convert not detecting empty changesets.
Francis Barber <fedora@barber-family.id.au>
parents: 8155
diff changeset
  1180
                    if not paths:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1181
                        self.ui.debug(b'revision %d has no entries\n' % revnum)
10618
508fda6b8637 convert: handle svn tree with empty roots (issue2079)
Patrick Mezard <pmezard@gmail.com>
parents: 10282
diff changeset
  1182
                        # If we ever leave the loop on an empty
508fda6b8637 convert: handle svn tree with empty roots (issue2079)
Patrick Mezard <pmezard@gmail.com>
parents: 10282
diff changeset
  1183
                        # revision, do not try to get a parent branch
508fda6b8637 convert: handle svn tree with empty roots (issue2079)
Patrick Mezard <pmezard@gmail.com>
parents: 10282
diff changeset
  1184
                        lastonbranch = lastonbranch or revnum == 0
5873
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
  1185
                        continue
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1186
                    cset, lastonbranch = parselogentry(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1187
                        paths, revnum, author, date, message
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1188
                    )
5873
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
  1189
                    if cset:
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
  1190
                        firstcset = cset
6173
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
  1191
                    if lastonbranch:
5873
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
  1192
                        break
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
  1193
            finally:
ecb4eb0cbff2 convert: make svn revision iterator interruptible
Patrick Mezard <pmezard@gmail.com>
parents: 5872
diff changeset
  1194
                stream.close()
5871
c1244685425a convert: fix parents of last fetched svn revision
Patrick Mezard <pmezard@gmail.com>
parents: 5870
diff changeset
  1195
6173
963000ed8cac convert: add shallow, single branch svn conversions via svn.startrev
Patrick Mezard <pmezard@gmail.com>
parents: 6172
diff changeset
  1196
            if not lastonbranch and firstcset and not firstcset.parents:
5871
c1244685425a convert: fix parents of last fetched svn revision
Patrick Mezard <pmezard@gmail.com>
parents: 5870
diff changeset
  1197
                # The first revision of the sequence (the last fetched one)
c1244685425a convert: fix parents of last fetched svn revision
Patrick Mezard <pmezard@gmail.com>
parents: 5870
diff changeset
  1198
                # has invalid parents if not a branch root. Find the parent
c1244685425a convert: fix parents of last fetched svn revision
Patrick Mezard <pmezard@gmail.com>
parents: 5870
diff changeset
  1199
                # revision now, if any.
c1244685425a convert: fix parents of last fetched svn revision
Patrick Mezard <pmezard@gmail.com>
parents: 5870
diff changeset
  1200
                try:
c1244685425a convert: fix parents of last fetched svn revision
Patrick Mezard <pmezard@gmail.com>
parents: 5870
diff changeset
  1201
                    firstrevnum = self.revnum(firstcset.rev)
c1244685425a convert: fix parents of last fetched svn revision
Patrick Mezard <pmezard@gmail.com>
parents: 5870
diff changeset
  1202
                    if firstrevnum > 1:
c1244685425a convert: fix parents of last fetched svn revision
Patrick Mezard <pmezard@gmail.com>
parents: 5870
diff changeset
  1203
                        latest = self.latest(self.module, firstrevnum - 1)
5957
971a17af5982 convert: prevent svn branches to leave the root module tree
Patrick Mezard <pmezard@gmail.com>
parents: 5956
diff changeset
  1204
                        if latest:
971a17af5982 convert: prevent svn branches to leave the root module tree
Patrick Mezard <pmezard@gmail.com>
parents: 5956
diff changeset
  1205
                            firstcset.parents.append(latest)
7381
b965605dfb2e convert: be even more tolerant when detecting svn tags
Patrick Mezard <pmezard@gmail.com>
parents: 7184
diff changeset
  1206
                except SvnPathNotFound:
5871
c1244685425a convert: fix parents of last fetched svn revision
Patrick Mezard <pmezard@gmail.com>
parents: 5870
diff changeset
  1207
                    pass
28460
d6dbb0968c13 convert: fix relative import of stdlib module in subversion
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 28459
diff changeset
  1208
        except svn.core.SubversionException as xxx_todo_changeme:
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24856
diff changeset
  1209
            (inst, num) = xxx_todo_changeme.args
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1210
            if num == svn.core.SVN_ERR_FS_NO_SUCH_REVISION:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1211
                raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1212
                    _(b'svn: branch has no revision %s') % to_revnum
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1213
                )
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1214
            raise
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1215
11134
33010ff1fd6f convert: merge sources getmode() into getfile()
Patrick Mezard <pmezard@gmail.com>
parents: 11133
diff changeset
  1216
    def getfile(self, file, rev):
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1217
        # TODO: ra.get_file transmits the whole file instead of diffs.
11127
ab575114ddd8 convert/svn: do not retrieve removed files
Patrick Mezard <pmezard@gmail.com>
parents: 11126
diff changeset
  1218
        if file in self.removed:
22296
650b5b6e75ed convert: use None value for missing files instead of overloading IOError
Mads Kiilerich <madski@unity3d.com>
parents: 22201
diff changeset
  1219
            return None, None
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1220
        try:
13690
af331f557942 convert/svn: extract revsplit() in a function
Patrick Mezard <pmezard@gmail.com>
parents: 13653
diff changeset
  1221
            new_module, revnum = revsplit(rev)[1:]
5872
7d4149cccc5d convert: fix cross-branches subversion revisions handling
Patrick Mezard <pmezard@gmail.com>
parents: 5871
diff changeset
  1222
            if self.module != new_module:
7d4149cccc5d convert: fix cross-branches subversion revisions handling
Patrick Mezard <pmezard@gmail.com>
parents: 5871
diff changeset
  1223
                self.module = new_module
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1224
                self.reparent(self.module)
28861
86db5cb55d46 pycompat: switch to util.stringio for py3 compat
timeless <timeless@mozdev.org>
parents: 28471
diff changeset
  1225
            io = stringio()
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1226
            info = svn.ra.get_file(self.ra, file, revnum, io)
7446
2d2e0be762be convert: work around svn.ra.get_files() not releasing input buffer
Patrick Mezard <pmezard@gmail.com>
parents: 7382
diff changeset
  1227
            data = io.getvalue()
17424
e7cfe3587ea4 fix trivial spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents: 17247
diff changeset
  1228
            # ra.get_file() seems to keep a reference on the input buffer
17479
17c3dbf3026f spelling: Explicitly
timeless@mozdev.org
parents: 17247
diff changeset
  1229
            # preventing collection. Release it explicitly.
7446
2d2e0be762be convert: work around svn.ra.get_files() not releasing input buffer
Patrick Mezard <pmezard@gmail.com>
parents: 7382
diff changeset
  1230
            io.close()
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1231
            if isinstance(info, list):
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1232
                info = info[-1]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1233
            mode = (b"svn:executable" in info) and b'x' or b''
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1234
            mode = (b"svn:special" in info) and b'l' or mode
28460
d6dbb0968c13 convert: fix relative import of stdlib module in subversion
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 28459
diff changeset
  1235
        except svn.core.SubversionException as e:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1236
            notfound = (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1237
                svn.core.SVN_ERR_FS_NOT_FOUND,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1238
                svn.core.SVN_ERR_RA_DAV_PATH_NOT_FOUND,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1239
            )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1240
            if e.apr_err in notfound:  # File not found
22296
650b5b6e75ed convert: use None value for missing files instead of overloading IOError
Mads Kiilerich <madski@unity3d.com>
parents: 22201
diff changeset
  1241
                return None, None
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1242
            raise
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1243
        if mode == b'l':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1244
            link_prefix = b"link "
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1245
            if data.startswith(link_prefix):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1246
                data = data[len(link_prefix) :]
4765
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1247
        return data, mode
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents:
diff changeset
  1248
11133
d7b6aab612b4 convert/svn: remove useless sort
Patrick Mezard <pmezard@gmail.com>
parents: 11132
diff changeset
  1249
    def _iterfiles(self, path, revnum):
d7b6aab612b4 convert/svn: remove useless sort
Patrick Mezard <pmezard@gmail.com>
parents: 11132
diff changeset
  1250
        """Enumerate all files in path at revnum, recursively."""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1251
        path = path.strip(b'/')
28460
d6dbb0968c13 convert: fix relative import of stdlib module in subversion
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 28459
diff changeset
  1252
        pool = svn.core.Pool()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1253
        rpath = b'/'.join([self.baseurl, quote(path)]).strip(b'/')
11167
b5ef95b5ae14 convert/svn: fix long line
Matt Mackall <mpm@selenic.com>
parents: 11137
diff changeset
  1254
        entries = svn.client.ls(rpath, optrev(revnum), True, self.ctx, pool)
13651
9777df929035 convert/svn: fix _iterfiles() output in root dir case (issue2647)
Patrick Mezard <pmezard@gmail.com>
parents: 13494
diff changeset
  1255
        if path:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1256
            path += b'/'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1257
        return (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1258
            (path + p)
43105
649d3ac37a12 py3: define and use pycompat.iteritems() for hgext/
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
  1259
            for p, e in pycompat.iteritems(entries)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1260
            if e.kind == svn.core.svn_node_file
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1261
        )
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1262
6539
7814d7bb77bc convert: rename get_entry_from_path() into an svn_source method
Patrick Mezard <pmezard@gmail.com>
parents: 6538
diff changeset
  1263
    def getrelpath(self, path, module=None):
7814d7bb77bc convert: rename get_entry_from_path() into an svn_source method
Patrick Mezard <pmezard@gmail.com>
parents: 6538
diff changeset
  1264
        if module is None:
7814d7bb77bc convert: rename get_entry_from_path() into an svn_source method
Patrick Mezard <pmezard@gmail.com>
parents: 6538
diff changeset
  1265
            module = self.module
7814d7bb77bc convert: rename get_entry_from_path() into an svn_source method
Patrick Mezard <pmezard@gmail.com>
parents: 6538
diff changeset
  1266
        # Given the repository url of this wc, say
7814d7bb77bc convert: rename get_entry_from_path() into an svn_source method
Patrick Mezard <pmezard@gmail.com>
parents: 6538
diff changeset
  1267
        #   "http://server/plone/CMFPlone/branches/Plone-2_0-branch"
7814d7bb77bc convert: rename get_entry_from_path() into an svn_source method
Patrick Mezard <pmezard@gmail.com>
parents: 6538
diff changeset
  1268
        # extract the "entry" portion (a relative path) from what
17424
e7cfe3587ea4 fix trivial spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents: 17247
diff changeset
  1269
        # svn log --xml says, i.e.
6539
7814d7bb77bc convert: rename get_entry_from_path() into an svn_source method
Patrick Mezard <pmezard@gmail.com>
parents: 6538
diff changeset
  1270
        #   "/CMFPlone/branches/Plone-2_0-branch/tests/PloneTestCase.py"
7814d7bb77bc convert: rename get_entry_from_path() into an svn_source method
Patrick Mezard <pmezard@gmail.com>
parents: 6538
diff changeset
  1271
        # that is to say "tests/PloneTestCase.py"
7814d7bb77bc convert: rename get_entry_from_path() into an svn_source method
Patrick Mezard <pmezard@gmail.com>
parents: 6538
diff changeset
  1272
        if path.startswith(module):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1273
            relative = path.rstrip(b'/')[len(module) :]
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1274
            if relative.startswith(b'/'):
6539
7814d7bb77bc convert: rename get_entry_from_path() into an svn_source method
Patrick Mezard <pmezard@gmail.com>
parents: 6538
diff changeset
  1275
                return relative[1:]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1276
            elif relative == b'':
6539
7814d7bb77bc convert: rename get_entry_from_path() into an svn_source method
Patrick Mezard <pmezard@gmail.com>
parents: 6538
diff changeset
  1277
                return relative
7814d7bb77bc convert: rename get_entry_from_path() into an svn_source method
Patrick Mezard <pmezard@gmail.com>
parents: 6538
diff changeset
  1278
7814d7bb77bc convert: rename get_entry_from_path() into an svn_source method
Patrick Mezard <pmezard@gmail.com>
parents: 6538
diff changeset
  1279
        # The path is outside our tracked tree...
44816
7a4630536e53 py3: avoid using %r format on bytes
Manuel Jacob <me@manueljacob.de>
parents: 44815
diff changeset
  1280
        self.ui.debug(
7a4630536e53 py3: avoid using %r format on bytes
Manuel Jacob <me@manueljacob.de>
parents: 44815
diff changeset
  1281
            b'%r is not under %r, ignoring\n'
7a4630536e53 py3: avoid using %r format on bytes
Manuel Jacob <me@manueljacob.de>
parents: 44815
diff changeset
  1282
            % (pycompat.bytestr(path), pycompat.bytestr(module))
7a4630536e53 py3: avoid using %r format on bytes
Manuel Jacob <me@manueljacob.de>
parents: 44815
diff changeset
  1283
        )
6539
7814d7bb77bc convert: rename get_entry_from_path() into an svn_source method
Patrick Mezard <pmezard@gmail.com>
parents: 6538
diff changeset
  1284
        return None
7814d7bb77bc convert: rename get_entry_from_path() into an svn_source method
Patrick Mezard <pmezard@gmail.com>
parents: 6538
diff changeset
  1285
11128
a9b427b5821e convert/svn: handle files/links replaced by dirs (issue2166)
Patrick Mezard <pmezard@gmail.com>
parents: 11127
diff changeset
  1286
    def _checkpath(self, path, revnum, module=None):
a9b427b5821e convert/svn: handle files/links replaced by dirs (issue2166)
Patrick Mezard <pmezard@gmail.com>
parents: 11127
diff changeset
  1287
        if module is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1288
            prevmodule = self.reparent(b'')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1289
            path = module + b'/' + path
11128
a9b427b5821e convert/svn: handle files/links replaced by dirs (issue2166)
Patrick Mezard <pmezard@gmail.com>
parents: 11127
diff changeset
  1290
        try:
a9b427b5821e convert/svn: handle files/links replaced by dirs (issue2166)
Patrick Mezard <pmezard@gmail.com>
parents: 11127
diff changeset
  1291
            # ra.check_path does not like leading slashes very much, it leads
a9b427b5821e convert/svn: handle files/links replaced by dirs (issue2166)
Patrick Mezard <pmezard@gmail.com>
parents: 11127
diff changeset
  1292
            # to PROPFIND subversion errors
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1293
            return svn.ra.check_path(self.ra, path.strip(b'/'), revnum)
11128
a9b427b5821e convert/svn: handle files/links replaced by dirs (issue2166)
Patrick Mezard <pmezard@gmail.com>
parents: 11127
diff changeset
  1294
        finally:
a9b427b5821e convert/svn: handle files/links replaced by dirs (issue2166)
Patrick Mezard <pmezard@gmail.com>
parents: 11127
diff changeset
  1295
            if module is not None:
a9b427b5821e convert/svn: handle files/links replaced by dirs (issue2166)
Patrick Mezard <pmezard@gmail.com>
parents: 11127
diff changeset
  1296
                self.reparent(prevmodule)
12770
614f0d8724ab check-code: find trailing whitespace
Martin Geisler <mg@lazybytes.net>
parents: 12343
diff changeset
  1297
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1298
    def _getlog(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1299
        self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1300
        paths,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1301
        start,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1302
        end,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1303
        limit=0,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1304
        discover_changed_paths=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1305
        strict_node_history=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1306
    ):
6850
c6bb8fae3bd2 convert: normalize paths sent to svn get_log (issue 1219)
Patrick Mezard <pmezard@gmail.com>
parents: 6848
diff changeset
  1307
        # Normalize path names, svn >= 1.5 only wants paths relative to
c6bb8fae3bd2 convert: normalize paths sent to svn get_log (issue 1219)
Patrick Mezard <pmezard@gmail.com>
parents: 6848
diff changeset
  1308
        # supplied URL
c6bb8fae3bd2 convert: normalize paths sent to svn get_log (issue 1219)
Patrick Mezard <pmezard@gmail.com>
parents: 6848
diff changeset
  1309
        relpaths = []
c6bb8fae3bd2 convert: normalize paths sent to svn get_log (issue 1219)
Patrick Mezard <pmezard@gmail.com>
parents: 6848
diff changeset
  1310
        for p in paths:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1311
            if not p.startswith(b'/'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1312
                p = self.module + b'/' + p
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1313
            relpaths.append(p.strip(b'/'))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1314
        args = [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1315
            self.baseurl,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1316
            relpaths,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1317
            start,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1318
            end,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1319
            limit,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1320
            discover_changed_paths,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1321
            strict_node_history,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1322
        ]
27314
a434f15dc0f4 convert/svn: quiet check-config
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
  1323
        # developer config: convert.svn.debugsvnlog
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1324
        if not self.ui.configbool(b'convert', b'svn.debugsvnlog'):
20420
62153c5d1ce2 convert: secret config option for disabling debugsvnlog
Mads Kiilerich <madski@unity3d.com>
parents: 20419
diff changeset
  1325
            return directlogstream(*args)
6850
c6bb8fae3bd2 convert: normalize paths sent to svn get_log (issue 1219)
Patrick Mezard <pmezard@gmail.com>
parents: 6848
diff changeset
  1326
        arg = encodeargs(args)
37123
a8a902d7176e procutil: bulk-replace function calls to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37087
diff changeset
  1327
        hgexe = procutil.hgexecutable()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1328
        cmd = b'%s debugsvnlog' % procutil.shellquote(hgexe)
37123
a8a902d7176e procutil: bulk-replace function calls to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37087
diff changeset
  1329
        stdin, stdout = procutil.popen2(procutil.quotecommand(cmd))
6850
c6bb8fae3bd2 convert: normalize paths sent to svn get_log (issue 1219)
Patrick Mezard <pmezard@gmail.com>
parents: 6848
diff changeset
  1330
        stdin.write(arg)
10071
661980567246 convert/svn: better handling of hg recursive call failure
Patrick Mezard <pmezard@gmail.com>
parents: 10051
diff changeset
  1331
        try:
661980567246 convert/svn: better handling of hg recursive call failure
Patrick Mezard <pmezard@gmail.com>
parents: 10051
diff changeset
  1332
            stdin.close()
661980567246 convert/svn: better handling of hg recursive call failure
Patrick Mezard <pmezard@gmail.com>
parents: 10051
diff changeset
  1333
        except IOError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1334
            raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1335
                _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1336
                    b'Mercurial failed to run itself, check'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1337
                    b' hg executable is in PATH'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1338
                )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1339
            )
6850
c6bb8fae3bd2 convert: normalize paths sent to svn get_log (issue 1219)
Patrick Mezard <pmezard@gmail.com>
parents: 6848
diff changeset
  1340
        return logstream(stdout)
c6bb8fae3bd2 convert: normalize paths sent to svn get_log (issue 1219)
Patrick Mezard <pmezard@gmail.com>
parents: 6848
diff changeset
  1341
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1342
38101
c560a4ea6dac py3: add b'' prefixes in hgext/convert/subversion.py
Pulkit Goyal <7895pulkit@gmail.com>
parents: 37123
diff changeset
  1343
pre_revprop_change = b'''#!/bin/sh
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1344
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1345
REPOS="$1"
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1346
REV="$2"
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1347
USER="$3"
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1348
PROPNAME="$4"
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1349
ACTION="$5"
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1350
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1351
if [ "$ACTION" = "M" -a "$PROPNAME" = "svn:log" ]; then exit 0; fi
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1352
if [ "$ACTION" = "A" -a "$PROPNAME" = "hg:convert-branch" ]; then exit 0; fi
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1353
if [ "$ACTION" = "A" -a "$PROPNAME" = "hg:convert-rev" ]; then exit 0; fi
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1354
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1355
echo "Changing prohibited revision property" >&2
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1356
exit 1
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1357
'''
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1358
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1359
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1360
class svn_sink(converter_sink, commandline):
38101
c560a4ea6dac py3: add b'' prefixes in hgext/convert/subversion.py
Pulkit Goyal <7895pulkit@gmail.com>
parents: 37123
diff changeset
  1361
    commit_re = re.compile(br'Committed revision (\d+).', re.M)
c560a4ea6dac py3: add b'' prefixes in hgext/convert/subversion.py
Pulkit Goyal <7895pulkit@gmail.com>
parents: 37123
diff changeset
  1362
    uuid_re = re.compile(br'Repository UUID:\s*(\S+)', re.M)
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1363
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1364
    def prerun(self):
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1365
        if self.wc:
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1366
            os.chdir(self.wc)
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1367
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1368
    def postrun(self):
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1369
        if self.wc:
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1370
            os.chdir(self.cwd)
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1371
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1372
    def join(self, name):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1373
        return os.path.join(self.wc, b'.svn', name)
5760
0145f9afb0e7 Removed tabs and trailing whitespace in python files
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5705
diff changeset
  1374
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1375
    def revmapfile(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1376
        return self.join(b'hg-shamap')
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1377
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1378
    def authorfile(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1379
        return self.join(b'hg-authormap')
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1380
35187
671aba341d90 convert: save an indicator of the repo type for sources and sinks
Matt Harbison <matt_harbison@yahoo.com>
parents: 34890
diff changeset
  1381
    def __init__(self, ui, repotype, path):
13480
69418d4525d1 convert/svn: abort operation when python bindings are not available
Azhagu Selvan SP <tamizhgeek@gmail.com>
parents: 13190
diff changeset
  1382
35187
671aba341d90 convert: save an indicator of the repo type for sources and sinks
Matt Harbison <matt_harbison@yahoo.com>
parents: 34890
diff changeset
  1383
        converter_sink.__init__(self, ui, repotype, path)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1384
        commandline.__init__(self, ui, b'svn')
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1385
        self.delete = []
5698
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1386
        self.setexec = []
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1387
        self.delexec = []
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1388
        self.copies = []
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1389
        self.wc = None
39823
24e493ec2229 py3: rename pycompat.getcwd() to encoding.getcwd() (API)
Matt Harbison <matt_harbison@yahoo.com>
parents: 39445
diff changeset
  1390
        self.cwd = encoding.getcwd()
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1391
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1392
        created = False
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1393
        if os.path.isfile(os.path.join(path, b'.svn', b'entries')):
17247
6d51a0c71d4e convert/svn: handle non-local svn destination paths (issue3142)
Patrick Mezard <patrick@mezard.eu>
parents: 17053
diff changeset
  1394
            self.wc = os.path.realpath(path)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1395
            self.run0(b'update')
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1396
        else:
44024
c1ccefb513e4 cleanup: drop redundant character escapes outside of `[]`
Matt Harbison <matt_harbison@yahoo.com>
parents: 43554
diff changeset
  1397
            if not re.search(br'^(file|http|https|svn|svn\+ssh)://', path):
17247
6d51a0c71d4e convert/svn: handle non-local svn destination paths (issue3142)
Patrick Mezard <patrick@mezard.eu>
parents: 17053
diff changeset
  1398
                path = os.path.realpath(path)
6d51a0c71d4e convert/svn: handle non-local svn destination paths (issue3142)
Patrick Mezard <patrick@mezard.eu>
parents: 17053
diff changeset
  1399
                if os.path.isdir(os.path.dirname(path)):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1400
                    if not os.path.exists(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1401
                        os.path.join(path, b'db', b'fs-type')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1402
                    ):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1403
                        ui.status(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1404
                            _(b"initializing svn repository '%s'\n")
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1405
                            % os.path.basename(path)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1406
                        )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1407
                        commandline(ui, b'svnadmin').run0(b'create', path)
17247
6d51a0c71d4e convert/svn: handle non-local svn destination paths (issue3142)
Patrick Mezard <patrick@mezard.eu>
parents: 17053
diff changeset
  1408
                        created = path
6d51a0c71d4e convert/svn: handle non-local svn destination paths (issue3142)
Patrick Mezard <patrick@mezard.eu>
parents: 17053
diff changeset
  1409
                    path = util.normpath(path)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1410
                    if not path.startswith(b'/'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1411
                        path = b'/' + path
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1412
                    path = b'file://' + path
5535
7501ef26402b convert: fix svn file:// URL generation under Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5528
diff changeset
  1413
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1414
            wcpath = os.path.join(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1415
                encoding.getcwd(), os.path.basename(path) + b'-wc'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1416
            )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1417
            ui.status(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1418
                _(b"initializing svn working copy '%s'\n")
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1419
                % os.path.basename(wcpath)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1420
            )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1421
            self.run0(b'checkout', path, wcpath)
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1422
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1423
            self.wc = wcpath
31256
b023caa6dbae vfs: use 'vfs' module directly in 'hgext.convert'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31226
diff changeset
  1424
        self.opener = vfsmod.vfs(self.wc)
b023caa6dbae vfs: use 'vfs' module directly in 'hgext.convert'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31226
diff changeset
  1425
        self.wopener = vfsmod.vfs(self.wc)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1426
        self.childmap = mapfile(ui, self.join(b'hg-childmap'))
24306
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 23583
diff changeset
  1427
        if util.checkexec(self.wc):
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 23583
diff changeset
  1428
            self.is_exec = util.isexec
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 23583
diff changeset
  1429
        else:
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 23583
diff changeset
  1430
            self.is_exec = None
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1431
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1432
        if created:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1433
            hook = os.path.join(created, b'hooks', b'pre-revprop-change')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1434
            fp = open(hook, b'wb')
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1435
            fp.write(pre_revprop_change)
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1436
            fp.close()
14232
df2399663392 rename util.set_flags to setflags
Adrian Buehlmann <adrian@cadifra.com>
parents: 14168
diff changeset
  1437
            util.setflags(hook, False, True)
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1438
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1439
        output = self.run0(b'info')
13530
5f69af0d2fb3 convert/svn: stop using svn bindings when pushing to svn
Patrick Mezard <pmezard@gmail.com>
parents: 13529
diff changeset
  1440
        self.uuid = self.uuid_re.search(output).group(1).strip()
5554
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5537
diff changeset
  1441
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1442
    def wjoin(self, *names):
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1443
        return os.path.join(self.wc, *names)
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1444
16511
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1445
    @propertycache
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1446
    def manifest(self):
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1447
        # As of svn 1.7, the "add" command fails when receiving
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1448
        # already tracked entries, so we have to track and filter them
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1449
        # ourselves.
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1450
        m = set()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1451
        output = self.run0(b'ls', recursive=True, xml=True)
16511
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1452
        doc = xml.dom.minidom.parseString(output)
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
  1453
        for e in doc.getElementsByTagName('entry'):
16511
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1454
            for n in e.childNodes:
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
  1455
                if n.nodeType != n.ELEMENT_NODE or n.tagName != 'name':
16511
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1456
                    continue
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
  1457
                name = ''.join(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1458
                    c.data for c in n.childNodes if c.nodeType == c.TEXT_NODE
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1459
                )
16511
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1460
                # Entries are compared with names coming from
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1461
                # mercurial, so bytes with undefined encoding. Our
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1462
                # best bet is to assume they are in local
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1463
                # encoding. They will be passed to command line calls
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1464
                # later anyway, so they better be.
31456
067add650129 encoding: factor out unicode variants of from/tolocal()
Yuya Nishihara <yuya@tcha.org>
parents: 31256
diff changeset
  1465
                m.add(encoding.unitolocal(name))
16511
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1466
                break
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1467
        return m
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1468
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1469
    def putfile(self, filename, flags, data):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1470
        if b'l' in flags:
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1471
            self.wopener.symlink(data, filename)
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1472
        else:
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1473
            try:
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1474
                if os.path.islink(self.wjoin(filename)):
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1475
                    os.unlink(filename)
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1476
            except OSError:
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1477
                pass
41628
36ee0d6d64c5 convert: handle exec bit removal while converting to svn
Nikita Slyusarev <nslus@yandex-team.com>
parents: 41627
diff changeset
  1478
36ee0d6d64c5 convert: handle exec bit removal while converting to svn
Nikita Slyusarev <nslus@yandex-team.com>
parents: 41627
diff changeset
  1479
            if self.is_exec:
36ee0d6d64c5 convert: handle exec bit removal while converting to svn
Nikita Slyusarev <nslus@yandex-team.com>
parents: 41627
diff changeset
  1480
                # We need to check executability of the file before the change,
36ee0d6d64c5 convert: handle exec bit removal while converting to svn
Nikita Slyusarev <nslus@yandex-team.com>
parents: 41627
diff changeset
  1481
                # because `vfs.write` is able to reset exec bit.
36ee0d6d64c5 convert: handle exec bit removal while converting to svn
Nikita Slyusarev <nslus@yandex-team.com>
parents: 41627
diff changeset
  1482
                wasexec = False
36ee0d6d64c5 convert: handle exec bit removal while converting to svn
Nikita Slyusarev <nslus@yandex-team.com>
parents: 41627
diff changeset
  1483
                if os.path.exists(self.wjoin(filename)):
36ee0d6d64c5 convert: handle exec bit removal while converting to svn
Nikita Slyusarev <nslus@yandex-team.com>
parents: 41627
diff changeset
  1484
                    wasexec = self.is_exec(self.wjoin(filename))
36ee0d6d64c5 convert: handle exec bit removal while converting to svn
Nikita Slyusarev <nslus@yandex-team.com>
parents: 41627
diff changeset
  1485
14168
135e244776f0 prevent transient leaks of file handle by using new helper functions
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 14152
diff changeset
  1486
            self.wopener.write(filename, data)
5536
fdee5d614fcc convert: force svn:executable when execute-bit is not supported
Patrick Mezard <pmezard@gmail.com>
parents: 5535
diff changeset
  1487
fdee5d614fcc convert: force svn:executable when execute-bit is not supported
Patrick Mezard <pmezard@gmail.com>
parents: 5535
diff changeset
  1488
            if self.is_exec:
41628
36ee0d6d64c5 convert: handle exec bit removal while converting to svn
Nikita Slyusarev <nslus@yandex-team.com>
parents: 41627
diff changeset
  1489
                if wasexec:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1490
                    if b'x' not in flags:
17031
f17b561cc473 convert: ignore svn:executable for subversion targets without exec bit support
Mads Kiilerich <mads@kiilerich.com>
parents: 16925
diff changeset
  1491
                        self.delexec.append(filename)
f17b561cc473 convert: ignore svn:executable for subversion targets without exec bit support
Mads Kiilerich <mads@kiilerich.com>
parents: 16925
diff changeset
  1492
                else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1493
                    if b'x' in flags:
17031
f17b561cc473 convert: ignore svn:executable for subversion targets without exec bit support
Mads Kiilerich <mads@kiilerich.com>
parents: 16925
diff changeset
  1494
                        self.setexec.append(filename)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1495
                util.setflags(self.wjoin(filename), False, b'x' in flags)
5698
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1496
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1497
    def _copyfile(self, source, dest):
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1498
        # SVN's copy command pukes if the destination file exists, but
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1499
        # our copyfile method expects to record a copy that has
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1500
        # already occurred.  Cross the semantic gap.
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1501
        wdest = self.wjoin(dest)
12343
28642f7fc2cf convert/svn: fix broken symlink renames in svn sink
Patrick Mezard <pmezard@gmail.com>
parents: 11779
diff changeset
  1502
        exists = os.path.lexists(wdest)
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1503
        if exists:
38197
aac4be30e250 py3: wrap tempfile.mkstemp() to use bytes path
Yuya Nishihara <yuya@tcha.org>
parents: 38101
diff changeset
  1504
            fd, tempname = pycompat.mkstemp(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1505
                prefix=b'hg-copy-', dir=os.path.dirname(wdest)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1506
            )
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1507
            os.close(fd)
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1508
            os.unlink(tempname)
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1509
            os.rename(wdest, tempname)
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1510
        try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1511
            self.run0(b'copy', source, dest)
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1512
        finally:
16511
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1513
            self.manifest.add(dest)
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1514
            if exists:
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1515
                try:
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1516
                    os.unlink(wdest)
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1517
                except OSError:
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1518
                    pass
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1519
                os.rename(tempname, wdest)
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1520
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1521
    def dirs_of(self, files):
8150
bbc24c0753a0 util: use built-in set and frozenset
Martin Geisler <mg@lazybytes.net>
parents: 8129
diff changeset
  1522
        dirs = set()
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1523
        for f in files:
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1524
            if os.path.isdir(self.wjoin(f)):
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1525
                dirs.add(f)
30610
c039eb03e652 convert: inline strutil.rfindall()
Yuya Nishihara <yuya@tcha.org>
parents: 30528
diff changeset
  1526
            i = len(f)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1527
            for i in iter(lambda: f.rfind(b'/', 0, i), -1):
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1528
                dirs.add(f[:i])
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1529
        return dirs
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1530
5698
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1531
    def add_dirs(self, files):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1532
        add_dirs = [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1533
            d for d in sorted(self.dirs_of(files)) if d not in self.manifest
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1534
        ]
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1535
        if add_dirs:
16511
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1536
            self.manifest.update(add_dirs)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1537
            self.xargs(add_dirs, b'add', non_recursive=True, quiet=True)
5698
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1538
        return add_dirs
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1539
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1540
    def add_files(self, files):
16511
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1541
        files = [f for f in files if f not in self.manifest]
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1542
        if files:
16511
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1543
            self.manifest.update(files)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1544
            self.xargs(files, b'add', quiet=True)
5698
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1545
        return files
5760
0145f9afb0e7 Removed tabs and trailing whitespace in python files
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5705
diff changeset
  1546
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1547
    def addchild(self, parent, child):
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1548
        self.childmap[parent] = child
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1549
5554
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5537
diff changeset
  1550
    def revid(self, rev):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1551
        return b"svn:%s@%s" % (self.uuid, rev)
5698
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1552
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1553
    def putcommit(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1554
        self, files, copies, parents, commit, source, revmap, full, cleanp2
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1555
    ):
15605
2ad5b8937d0d convert/svn: update svn working copy only when necessary
Patrick Mezard <pmezard@gmail.com>
parents: 15599
diff changeset
  1556
        for parent in parents:
2ad5b8937d0d convert/svn: update svn working copy only when necessary
Patrick Mezard <pmezard@gmail.com>
parents: 15599
diff changeset
  1557
            try:
2ad5b8937d0d convert/svn: update svn working copy only when necessary
Patrick Mezard <pmezard@gmail.com>
parents: 15599
diff changeset
  1558
                return self.revid(self.childmap[parent])
2ad5b8937d0d convert/svn: update svn working copy only when necessary
Patrick Mezard <pmezard@gmail.com>
parents: 15599
diff changeset
  1559
            except KeyError:
2ad5b8937d0d convert/svn: update svn working copy only when necessary
Patrick Mezard <pmezard@gmail.com>
parents: 15599
diff changeset
  1560
                pass
2ad5b8937d0d convert/svn: update svn working copy only when necessary
Patrick Mezard <pmezard@gmail.com>
parents: 15599
diff changeset
  1561
6716
c9b8d2565b92 convert: reintegrate file retrieval code in sinks
Patrick Mezard <pmezard@gmail.com>
parents: 6557
diff changeset
  1562
        # Apply changes to working copy
c9b8d2565b92 convert: reintegrate file retrieval code in sinks
Patrick Mezard <pmezard@gmail.com>
parents: 6557
diff changeset
  1563
        for f, v in files:
22296
650b5b6e75ed convert: use None value for missing files instead of overloading IOError
Mads Kiilerich <madski@unity3d.com>
parents: 22201
diff changeset
  1564
            data, mode = source.getfile(f, v)
650b5b6e75ed convert: use None value for missing files instead of overloading IOError
Mads Kiilerich <madski@unity3d.com>
parents: 22201
diff changeset
  1565
            if data is None:
6716
c9b8d2565b92 convert: reintegrate file retrieval code in sinks
Patrick Mezard <pmezard@gmail.com>
parents: 6557
diff changeset
  1566
                self.delete.append(f)
c9b8d2565b92 convert: reintegrate file retrieval code in sinks
Patrick Mezard <pmezard@gmail.com>
parents: 6557
diff changeset
  1567
            else:
11134
33010ff1fd6f convert: merge sources getmode() into getfile()
Patrick Mezard <pmezard@gmail.com>
parents: 11133
diff changeset
  1568
                self.putfile(f, mode, data)
6716
c9b8d2565b92 convert: reintegrate file retrieval code in sinks
Patrick Mezard <pmezard@gmail.com>
parents: 6557
diff changeset
  1569
                if f in copies:
c9b8d2565b92 convert: reintegrate file retrieval code in sinks
Patrick Mezard <pmezard@gmail.com>
parents: 6557
diff changeset
  1570
                    self.copies.append([copies[f], f])
22300
35ab037de989 convert: introduce --full for converting all files
Mads Kiilerich <madski@unity3d.com>
parents: 22298
diff changeset
  1571
        if full:
35ab037de989 convert: introduce --full for converting all files
Mads Kiilerich <madski@unity3d.com>
parents: 22298
diff changeset
  1572
            self.delete.extend(sorted(self.manifest.difference(files)))
6716
c9b8d2565b92 convert: reintegrate file retrieval code in sinks
Patrick Mezard <pmezard@gmail.com>
parents: 6557
diff changeset
  1573
        files = [f[0] for f in files]
c9b8d2565b92 convert: reintegrate file retrieval code in sinks
Patrick Mezard <pmezard@gmail.com>
parents: 6557
diff changeset
  1574
8150
bbc24c0753a0 util: use built-in set and frozenset
Martin Geisler <mg@lazybytes.net>
parents: 8129
diff changeset
  1575
        entries = set(self.delete)
bbc24c0753a0 util: use built-in set and frozenset
Martin Geisler <mg@lazybytes.net>
parents: 8129
diff changeset
  1576
        files = frozenset(files)
5698
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1577
        entries.update(self.add_dirs(files.difference(entries)))
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1578
        if self.copies:
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1579
            for s, d in self.copies:
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1580
                self._copyfile(s, d)
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1581
            self.copies = []
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1582
        if self.delete:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1583
            self.xargs(self.delete, b'delete')
16511
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1584
            for f in self.delete:
ecd2fbe68b25 convert/svn: make svn sink work with svn 1.7
Patrick Mezard <patrick@mezard.eu>
parents: 16466
diff changeset
  1585
                self.manifest.remove(f)
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1586
            self.delete = []
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1587
        entries.update(self.add_files(files.difference(entries)))
5698
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1588
        if self.delexec:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1589
            self.xargs(self.delexec, b'propdel', b'svn:executable')
5698
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1590
            self.delexec = []
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1591
        if self.setexec:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1592
            self.xargs(self.setexec, b'propset', b'svn:executable', b'*')
5698
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1593
            self.setexec = []
b63ef7b1441c convert: svn-sink: copy and set properties after adding dirs/files
Maxim Dounin <mdounin@mdounin.ru>
parents: 5618
diff changeset
  1594
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1595
        fd, messagefile = pycompat.mkstemp(prefix=b'hg-convert-')
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
  1596
        fp = os.fdopen(fd, 'wb')
36186
0f9e52f900c4 convert: fix line ending of mapfile and commit.desc file
Yuya Nishihara <yuya@tcha.org>
parents: 36170
diff changeset
  1597
        fp.write(util.tonativeeol(commit.desc))
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1598
        fp.close()
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1599
        try:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1600
            output = self.run0(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1601
                b'commit',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1602
                username=stringutil.shortuser(commit.author),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1603
                file=messagefile,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1604
                encoding=b'utf-8',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1605
            )
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1606
            try:
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1607
                rev = self.commit_re.search(output).group(1)
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1608
            except AttributeError:
41627
2c13e91ede6e convert: handle empty intial commits while converting to svn
Nikita Slyusarev <nslus@yandex-team.com>
parents: 41387
diff changeset
  1609
                if not files:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1610
                    return parents[0] if parents else b'None'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1611
                self.ui.warn(_(b'unexpected svn output:\n'))
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1612
                self.ui.warn(output)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1613
                raise error.Abort(_(b'unable to cope with svn output'))
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1614
            if commit.rev:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1615
                self.run(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1616
                    b'propset',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1617
                    b'hg:convert-rev',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1618
                    commit.rev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1619
                    revprop=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1620
                    revision=rev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1621
                )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1622
            if commit.branch and commit.branch != b'default':
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1623
                self.run(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1624
                    b'propset',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1625
                    b'hg:convert-branch',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1626
                    commit.branch,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1627
                    revprop=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1628
                    revision=rev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1629
                )
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1630
            for parent in parents:
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1631
                self.addchild(parent, rev)
5554
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5537
diff changeset
  1632
            return self.revid(rev)
5513
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1633
        finally:
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1634
            os.unlink(messagefile)
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1635
f0c58fd4b798 convert: add support for Subversion as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5511
diff changeset
  1636
    def puttags(self, tags):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1637
        self.ui.warn(_(b'writing Subversion tags is not yet implemented\n'))
11778
7e5f5e5858f9 convert: Using --dest-type svn crashed, if the source repo used tags.
Daniel J. Lauk <daniel.lauk@gmail.com>
parents: 11196
diff changeset
  1638
        return None, None
16106
d75aa756149b convert: use splicemap entries when sorting revisions (issue1748)
Patrick Mezard <patrick@mezard.eu>
parents: 16067
diff changeset
  1639
21635
5f2cc464e502 convert: introduce hascommitfrommap sink method
Mads Kiilerich <madski@unity3d.com>
parents: 21634
diff changeset
  1640
    def hascommitfrommap(self, rev):
5f2cc464e502 convert: introduce hascommitfrommap sink method
Mads Kiilerich <madski@unity3d.com>
parents: 21634
diff changeset
  1641
        # We trust that revisions referenced in a map still is present
5f2cc464e502 convert: introduce hascommitfrommap sink method
Mads Kiilerich <madski@unity3d.com>
parents: 21634
diff changeset
  1642
        # TODO: implement something better if necessary and feasible
5f2cc464e502 convert: introduce hascommitfrommap sink method
Mads Kiilerich <madski@unity3d.com>
parents: 21634
diff changeset
  1643
        return True
5f2cc464e502 convert: introduce hascommitfrommap sink method
Mads Kiilerich <madski@unity3d.com>
parents: 21634
diff changeset
  1644
21634
23b24d6a70c8 convert: rename sink hascommit to hascommitforsplicemap
Mads Kiilerich <madski@unity3d.com>
parents: 21076
diff changeset
  1645
    def hascommitforsplicemap(self, rev):
16106
d75aa756149b convert: use splicemap entries when sorting revisions (issue1748)
Patrick Mezard <patrick@mezard.eu>
parents: 16067
diff changeset
  1646
        # This is not correct as one can convert to an existing subversion
d75aa756149b convert: use splicemap entries when sorting revisions (issue1748)
Patrick Mezard <patrick@mezard.eu>
parents: 16067
diff changeset
  1647
        # repository and childmap would not list all revisions. Too bad.
d75aa756149b convert: use splicemap entries when sorting revisions (issue1748)
Patrick Mezard <patrick@mezard.eu>
parents: 16067
diff changeset
  1648
        if rev in self.childmap:
d75aa756149b convert: use splicemap entries when sorting revisions (issue1748)
Patrick Mezard <patrick@mezard.eu>
parents: 16067
diff changeset
  1649
            return True
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1650
        raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1651
            _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1652
                b'splice map revision %s not found in subversion '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1653
                b'child map (revision lookups are not implemented)'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1654
            )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1655
            % rev
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42419
diff changeset
  1656
        )