Mercurial > hg
annotate doc/check-seclevel.py @ 30760:753b9d43ca81
internals: document compression negotiation
As part of adding zstd support to all of the things, we'll need
to teach the wire protocol to support non-zlib compression formats.
This commit documents how we'll implement that.
To understand how we arrived at this proposal, let's look at how
things are done today.
The wire protocol today doesn't have a unified format. Instead,
there is a limited facility for differentiating replies as successful
or not. And, each command essentially defines its own response format.
A significant deficiency in the current protocol is the lack of
payload framing over the SSH transport. In the HTTP transport,
chunked transfer is used and the end of an HTTP response body (and
the end of a Mercurial command response) can be identified by a 0
length chunk. This is how HTTP chunked transfer works. But in the
SSH transport, there is no such framing, at least for certain
responses (notably the response to "getbundle" requests). Clients
can't simply read until end of stream because the socket is
persistent and reused for multiple requests. Clients need to know
when they've encountered the end of a request but there is nothing
simple for them to key off of to detect this. So what happens is
the client must decode the payload (as opposed to being dumb and
forwarding frames/packets). This means the payload itself needs
to support identifying end of stream. In some cases (bundle2), it
also means the payload can encode "error" or "interrupt" events
telling the client to e.g. abort processing. The lack of framing
on the SSH transport and the transfer of its responsibilities to
e.g. bundle2 is a massive layering violation and a wart on the
protocol architecture. It needs to be fixed someday by inventing a
proper framing protocol.
So about compression.
The client transport abstractions have a "_callcompressable()"
API. This API is called to invoke a remote command that will
send a compressible response. The response is essentially a
"streaming" response (no framing data at the Mercurial layer)
that is fed into a decompressor.
On the HTTP transport, the decompressor is zlib and only zlib.
There is currently no mechanism for the client to specify an
alternate compression format. And, clients don't advertise what
compression formats they support or ask the server to send a
specific compression format. Instead, it is assumed that non-error
responses to "compressible" commands are zlib compressed.
On the SSH transport, there is no compression at the Mercurial
protocol layer. Instead, compression must be handled by SSH
itself (e.g. `ssh -C`) or within the payload data (e.g. bundle
compression).
For the HTTP transport, adding new compression formats is pretty
straightforward. Once you know what decompressor to use, you can
stream data into the decompressor until you reach a 0 size HTTP
chunk, at which point you are at end of stream.
So our wire protocol changes for the HTTP transport are pretty
straightforward: the client and server advertise what compression
formats they support and an appropriate compression format is
chosen. We introduce a new HTTP media type to hold compressed
payloads. The header of the payload defines the compression format
being used. Whoever is on the receiving end can sniff the first few
bytes route to an appropriate decompressor.
Support for multiple compression formats is advertised on both
server and client. The server advertises a "compression" capability
saying which compression formats it supports and in what order they
are preferred. Clients advertise their support for multiple
compression formats and media types via the introduced "X-HgProto"
request header.
Strictly speaking, servers don't need to advertise which compression
formats they support. But doing so allows clients to fail fast if
they don't support any of the formats the server does. This is useful
in situations like sending bundles, where the client may have to
perform expensive computation before sending data to the server.
Rather than simply advertise a list of supported compression formats,
we introduce an additional "httpmediatype" server capability
advertising which media types the server supports. This means servers
are explicit about what formats they exchange. IMO, this is superior
to inferring support from other capabilities (like "compression").
By advertising compression support on each request in the "X-HgProto"
header and media type and direction at the server level, we are able
to gradually transition existing commands/responses to the new media
type and possibly compression. Contrast with the old world, where we
only supported a single media type and the use of compression was
built-in to the semantics of the command on both client and server.
In the new world, if "application/mercurial-0.2" is supported,
compression is supported. It's that simple.
It's worth noting that we explicitly don't use "Accept,"
"Accept-Encoding," "Content-Encoding," or "Transfer-Encoding" for
content negotiation and compression. People knowledgeable of the HTTP
specifications will say that we should use these because that's
what they are designed to be used for. They have a point and I
sympathize with the argument. Earlier versions of this commit even
defined supported media types in the "Accept" header. However, my
years of experience rolling out services leveraging HTTP has taught
me to not trust the HTTP layer, especially if you are going outside
the normal spec (such as using a custom "Content-Encoding" value to
represent zstd streams). I've seen load balancers, proxies, and other
network devices do very bad and unexpected things to HTTP messages
(like insisting zlib compressed content is decoded and then re-encoded
at a different compression level or even stripping compression
completely). I've found that the best way to avoid surprises when
writing protocols on top of HTTP is to use HTTP as a dumb transport as
much as possible to minimize the chances that an "intelligent" agent
between endpoints will muck with your data. While the widespread use of
TLS is mitigating many intermediate network agents interfering with
HTTP, there are still problems at the edges, with e.g. the origin HTTP
server needing to convert HTTP to and from WSGI and buggy or
feature-lacking HTTP client implementations. I've found the best way to
avoid these problems is to avoid using headers like "Content-Encoding"
and to bake as much logic as possible into media types and HTTP message
bodies. The protocol changes in this commit do rely on a custom HTTP
request header and the "Content-Type" headers. But we used them before,
so we shouldn't be increasing our exposure to "bad" HTTP agents.
For the SSH transport, we can't easily implement content negotiation
to determine compression formats because the SSH transport has no
content negotiation capabilities today. And without a framing protocol,
we don't know how much data to feed into a decompressor. So in order
to implement compression support on the SSH transport, we'd need to
invent a mechanism to represent content types and an outer framing
protocol to stream data robustly. While I'm fully capable of doing
that, it is a lot of work and not something that should be undertaken
lightly. My opinion is that if we're going to change the SSH transport
protocol, we should take a long hard look at implementing a grand
unified protocol that attempts to address all the deficiencies with
the existing protocol. While I want this to happen, that would be
massive scope bloat standing in the way of zstd support. So, I've
decided to take the easy solution: the SSH transport will not gain
support for multiple compression formats. Keep in mind it doesn't
support *any* compression today. So essentially nothing is changing
on the SSH front.
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Sat, 24 Dec 2016 13:56:36 -0700 |
parents | d83ca854fa21 |
children | e9f456183402 |
rev | line source |
---|---|
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
1 #!/usr/bin/env python |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
2 # |
26192
67e6e55360d2
check-seclevel: fix file description grammar
timeless@mozdev.org
parents:
21792
diff
changeset
|
3 # checkseclevel - checking section title levels in each online help document |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
4 |
28965
98153441c8cc
py3: make check-seclevel use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
27770
diff
changeset
|
5 from __future__ import absolute_import |
98153441c8cc
py3: make check-seclevel use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
27770
diff
changeset
|
6 |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
7 import optparse |
28965
98153441c8cc
py3: make check-seclevel use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
27770
diff
changeset
|
8 import os |
98153441c8cc
py3: make check-seclevel use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
27770
diff
changeset
|
9 import sys |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
10 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
11 # import from the live mercurial repo |
27221
ab776610fc6d
check-seclevel: set module load policy to Python only
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26413
diff
changeset
|
12 os.environ['HGMODULEPOLICY'] = 'py' |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
13 sys.path.insert(0, "..") |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
14 from mercurial import demandimport; demandimport.enable() |
28965
98153441c8cc
py3: make check-seclevel use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
27770
diff
changeset
|
15 from mercurial import ( |
98153441c8cc
py3: make check-seclevel use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
27770
diff
changeset
|
16 commands, |
98153441c8cc
py3: make check-seclevel use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
27770
diff
changeset
|
17 extensions, |
98153441c8cc
py3: make check-seclevel use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
27770
diff
changeset
|
18 help, |
98153441c8cc
py3: make check-seclevel use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
27770
diff
changeset
|
19 minirst, |
98153441c8cc
py3: make check-seclevel use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
27770
diff
changeset
|
20 ui as uimod, |
98153441c8cc
py3: make check-seclevel use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
27770
diff
changeset
|
21 ) |
98153441c8cc
py3: make check-seclevel use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
27770
diff
changeset
|
22 |
98153441c8cc
py3: make check-seclevel use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
27770
diff
changeset
|
23 table = commands.table |
98153441c8cc
py3: make check-seclevel use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
27770
diff
changeset
|
24 helptable = help.helptable |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
25 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
26 level2mark = ['"', '=', '-', '.', '#'] |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
27 reservedmarks = ['"'] |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
28 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
29 mark2level = {} |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
30 for m, l in zip(level2mark, xrange(len(level2mark))): |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
31 if m not in reservedmarks: |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
32 mark2level[m] = l |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
33 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
34 initlevel_topic = 0 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
35 initlevel_cmd = 1 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
36 initlevel_ext = 1 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
37 initlevel_ext_cmd = 3 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
38 |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
39 def showavailables(ui, initlevel): |
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
40 ui.warn((' available marks and order of them in this help: %s\n') % |
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
41 (', '.join(['%r' % (m * 4) for m in level2mark[initlevel + 1:]]))) |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
42 |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
43 def checkseclevel(ui, doc, name, initlevel): |
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
44 ui.note(('checking "%s"\n') % name) |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
45 blocks, pruned = minirst.parse(doc, 0, ['verbose']) |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
46 errorcnt = 0 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
47 curlevel = initlevel |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
48 for block in blocks: |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
49 if block['type'] != 'section': |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
50 continue |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
51 mark = block['underline'] |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
52 title = block['lines'][0] |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
53 if (mark not in mark2level) or (mark2level[mark] <= initlevel): |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
54 ui.warn(('invalid section mark %r for "%s" of %s\n') % |
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
55 (mark * 4, title, name)) |
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
56 showavailables(ui, initlevel) |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
57 errorcnt += 1 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
58 continue |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
59 nextlevel = mark2level[mark] |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
60 if curlevel < nextlevel and curlevel + 1 != nextlevel: |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
61 ui.warn(('gap of section level at "%s" of %s\n') % |
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
62 (title, name)) |
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
63 showavailables(ui, initlevel) |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
64 errorcnt += 1 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
65 continue |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
66 ui.note(('appropriate section level for "%s %s"\n') % |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
67 (mark * (nextlevel * 2), title)) |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
68 curlevel = nextlevel |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
69 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
70 return errorcnt |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
71 |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
72 def checkcmdtable(ui, cmdtable, namefmt, initlevel): |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
73 errorcnt = 0 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
74 for k, entry in cmdtable.items(): |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
75 name = k.split("|")[0].lstrip("^") |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
76 if not entry[0].__doc__: |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
77 ui.note(('skip checking %s: no help document\n') % |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
78 (namefmt % name)) |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
79 continue |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
80 errorcnt += checkseclevel(ui, entry[0].__doc__, |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
81 namefmt % name, |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
82 initlevel) |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
83 return errorcnt |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
84 |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
85 def checkhghelps(ui): |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
86 errorcnt = 0 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
87 for names, sec, doc in helptable: |
21792
e15c991fe2ec
check-seclevel: restore use of callable() since it was readded in Python 3.2
Augie Fackler <raf@durin42.com>
parents:
17648
diff
changeset
|
88 if callable(doc): |
26413
e0c572d4d112
help: pass around ui to doc loader (API)
Yuya Nishihara <yuya@tcha.org>
parents:
26411
diff
changeset
|
89 doc = doc(ui) |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
90 errorcnt += checkseclevel(ui, doc, |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
91 '%s help topic' % names[0], |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
92 initlevel_topic) |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
93 |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
94 errorcnt += checkcmdtable(ui, table, '%s command', initlevel_cmd) |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
95 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
96 for name in sorted(extensions.enabled().keys() + |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
97 extensions.disabled().keys()): |
27511
44a596a8bed1
check-seclevel: pass a ui to the extension loader
Bryan O'Sullivan <bos@serpentine.com>
parents:
27510
diff
changeset
|
98 mod = extensions.load(ui, name, None) |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
99 if not mod.__doc__: |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
100 ui.note(('skip checking %s extension: no help document\n') % name) |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
101 continue |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
102 errorcnt += checkseclevel(ui, mod.__doc__, |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
103 '%s extension' % name, |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
104 initlevel_ext) |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
105 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
106 cmdtable = getattr(mod, 'cmdtable', None) |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
107 if cmdtable: |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
108 errorcnt += checkcmdtable(ui, cmdtable, |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
109 '%s command of ' + name + ' extension', |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
110 initlevel_ext_cmd) |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
111 return errorcnt |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
112 |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
113 def checkfile(ui, filename, initlevel): |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
114 if filename == '-': |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
115 filename = 'stdin' |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
116 doc = sys.stdin.read() |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
117 else: |
27770
1b8c7d59be43
check-seclevel: use a context manager for file I/O
Bryan O'Sullivan <bryano@fb.com>
parents:
27511
diff
changeset
|
118 with open(filename) as fp: |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
119 doc = fp.read() |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
120 |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
121 ui.note(('checking input from %s with initlevel %d\n') % |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
122 (filename, initlevel)) |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
123 return checkseclevel(ui, doc, 'input from %s' % filename, initlevel) |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
124 |
26398
70abba798098
check-seclevel: wrap entry point by function
Yuya Nishihara <yuya@tcha.org>
parents:
26192
diff
changeset
|
125 def main(): |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
126 optparser = optparse.OptionParser("""%prog [options] |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
127 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
128 This checks all help documents of Mercurial (topics, commands, |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
129 extensions and commands of them), if no file is specified by --file |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
130 option. |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
131 """) |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
132 optparser.add_option("-v", "--verbose", |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
133 help="enable additional output", |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
134 action="store_true") |
27510
020d93c4a727
check-seclevel: add a --debug option
Bryan O'Sullivan <bos@serpentine.com>
parents:
27221
diff
changeset
|
135 optparser.add_option("-d", "--debug", |
020d93c4a727
check-seclevel: add a --debug option
Bryan O'Sullivan <bos@serpentine.com>
parents:
27221
diff
changeset
|
136 help="debug mode", |
020d93c4a727
check-seclevel: add a --debug option
Bryan O'Sullivan <bos@serpentine.com>
parents:
27221
diff
changeset
|
137 action="store_true") |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
138 optparser.add_option("-f", "--file", |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
139 help="filename to read in (or '-' for stdin)", |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
140 action="store", default="") |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
141 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
142 optparser.add_option("-t", "--topic", |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
143 help="parse file as help topic", |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
144 action="store_const", dest="initlevel", const=0) |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
145 optparser.add_option("-c", "--command", |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
146 help="parse file as help of core command", |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
147 action="store_const", dest="initlevel", const=1) |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
148 optparser.add_option("-e", "--extension", |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
149 help="parse file as help of extension", |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
150 action="store_const", dest="initlevel", const=1) |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
151 optparser.add_option("-C", "--extension-command", |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
152 help="parse file as help of extension command", |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
153 action="store_const", dest="initlevel", const=3) |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
154 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
155 optparser.add_option("-l", "--initlevel", |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
156 help="set initial section level manually", |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
157 action="store", type="int", default=0) |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
158 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
159 (options, args) = optparser.parse_args() |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
160 |
30559
d83ca854fa21
ui: factor out ui.load() to create a ui without loading configs (API)
Yuya Nishihara <yuya@tcha.org>
parents:
28965
diff
changeset
|
161 ui = uimod.ui.load() |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
162 ui.setconfig('ui', 'verbose', options.verbose, '--verbose') |
27510
020d93c4a727
check-seclevel: add a --debug option
Bryan O'Sullivan <bos@serpentine.com>
parents:
27221
diff
changeset
|
163 ui.setconfig('ui', 'debug', options.debug, '--debug') |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
164 |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
165 if options.file: |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
166 if checkfile(ui, options.file, options.initlevel): |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
167 sys.exit(1) |
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
168 else: |
26411
dd62eaa82cbe
check-seclevel: use ui to show status and error messages
Yuya Nishihara <yuya@tcha.org>
parents:
26398
diff
changeset
|
169 if checkhghelps(ui): |
17648
07f1ac17b722
doc: add the tool to check section marks in help documents
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
170 sys.exit(1) |
26398
70abba798098
check-seclevel: wrap entry point by function
Yuya Nishihara <yuya@tcha.org>
parents:
26192
diff
changeset
|
171 |
70abba798098
check-seclevel: wrap entry point by function
Yuya Nishihara <yuya@tcha.org>
parents:
26192
diff
changeset
|
172 if __name__ == "__main__": |
70abba798098
check-seclevel: wrap entry point by function
Yuya Nishihara <yuya@tcha.org>
parents:
26192
diff
changeset
|
173 main() |