author | Matt Harbison <matt_harbison@yahoo.com> |
Fri, 06 Jan 2023 00:05:14 -0500 | |
changeset 49865 | 5392bf258049 |
parent 44274 | 87167caa2989 |
permissions | -rw-r--r-- |
42242 | 1 |
Prior to removing (EXPERIMENTAL) |
2 |
-------------------------------- |
|
3 |
||
4 |
These things affect UI and/or behavior, and should probably be implemented (or |
|
5 |
ruled out) prior to taking off the experimental shrinkwrap. |
|
6 |
||
7 |
#. Finish the `hg convert` story |
|
8 |
||
9 |
* Add an argument to accept a rules file to apply during conversion? |
|
10 |
Currently `lfs.track` is the only way to affect the conversion. |
|
11 |
* drop `lfs.track` config settings |
|
12 |
* splice in `.hglfs` file for normal repo -> lfs conversions? |
|
13 |
||
14 |
#. Stop uploading blobs when pushing between local repos |
|
15 |
||
16 |
* Could probably hardlink directly to the other local repo's store |
|
17 |
* Support inferring `lfs.url` for local push/pull (currently only supports |
|
18 |
http) |
|
19 |
||
20 |
#. Stop uploading blobs on strip/amend/histedit/etc. |
|
21 |
||
22 |
* This seems to be a side effect of doing it for `hg bundle`, which probably |
|
23 |
makes sense. |
|
24 |
||
25 |
#. Handle a server with the extension loaded and a client without the extension |
|
26 |
more gracefully. |
|
27 |
||
28 |
* `changegroup3` is still experimental, and not enabled by default. |
|
29 |
* Figure out how to `introduce LFS to the server repo |
|
30 |
<https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-September/122281.html>`_. |
|
31 |
See the TODO in test-lfs-serve.t. |
|
32 |
||
33 |
#. Remove `lfs.retry` hack in client? This came from FB, but it's not clear why |
|
34 |
it is/was needed. |
|
35 |
||
36 |
#. `hg export` currently writes out the LFS blob. Should it write the pointer |
|
37 |
instead? |
|
38 |
||
39 |
* `hg diff` is similar, and probably shouldn't see the pointer file |
|
40 |
||
41 |
#. Show to-be-applied rules with `hg files -r 'wdir()' 'set:lfs()'` |
|
42 |
||
43 |
* `debugignore` can show file + line number, so a dedicated command could be |
|
44 |
useful too. |
|
45 |
||
46 |
#. Filesets, revsets and templates |
|
47 |
||
48 |
* A dedicated revset should be faster than `'file(set:lfs())'` |
|
49 |
* Attach `{lfsoid}` and `{lfspointer}` to `general keywords |
|
50 |
<https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-January/110251.html>`_, |
|
51 |
IFF the file is a blob |
|
52 |
* Drop existing items that would be redundant with general support |
|
53 |
||
54 |
#. Can `grep` avoid downloading most things? |
|
55 |
||
56 |
* Add a command option to skip LFS blobs? |
|
57 |
||
58 |
#. Add a flag that's visible in `hg files -v` to indicate external storage? |
|
59 |
||
60 |
#. Server side issues |
|
61 |
||
62 |
* Check for local disk space before allowing upload. (I've got a patch for |
|
63 |
this.) |
|
64 |
* Make sure the http codes used are appropriate. |
|
65 |
* `Why is copying the Authorization header into the JSON payload necessary |
|
66 |
<https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-April/116230.html>`_? |
|
67 |
* `LFS-Authenticate` header support in client and server(?) |
|
68 |
||
69 |
#. Add locks on cache and blob store |
|
70 |
||
71 |
* This is complicated with a global store, and multiple potentially unrelated |
|
72 |
local repositories that reference the same blob. |
|
73 |
* Alternately, maybe just handle collisions when trying to create the same |
|
74 |
blob in the store somehow. |
|
75 |
||
76 |
#. Are proper file sizes reported in `debugupgraderepo`? |
|
77 |
||
78 |
#. Finish prefetching files |
|
79 |
||
42479
ed9a99564051
lfs: correct an error in the TODO file
Matt Harbison <matt_harbison@yahoo.com>
parents:
42242
diff
changeset
|
80 |
* `-T {data}` (other than cat?) |
42242 | 81 |
* `verify` |
82 |
* `grep` |
|
83 |
||
84 |
#. Output cleanup |
|
85 |
||
86 |
* Can we print the url when connecting to the blobstore? (A sudden |
|
87 |
connection refused after pulling commits looks confusing.) Problem is, |
|
88 |
'pushing to main url' is printed, and then lfs wants to upload before going |
|
89 |
back to the main repo transfer, so then *that* could be confusing with |
|
90 |
extra output. (This is kinda improved with 380f5131ee7b and 9f78d10742af.) |
|
91 |
||
92 |
* Add more progress indicators? Uploading a large repo looks idle for a long |
|
93 |
time while it scans for blobs in each outgoing revision. |
|
94 |
||
95 |
* Print filenames instead of hashes in error messages |
|
96 |
||
97 |
* subrepo aware paths, where necessary |
|
98 |
||
99 |
* Is existing output at the right status/note/debug level? |
|
100 |
||
101 |
#. Can `verify` be done without downloading everything? |
|
102 |
||
103 |
* If we know that we are talking to an hg server, we can leverage the fact |
|
104 |
that it validates in the Batch API portion, and skip d/l altogether. OTOH, |
|
105 |
maybe we should download the files unconditionally for forensics. The |
|
106 |
alternative is to define a custom transfer handler that definitively |
|
107 |
verifies without transferring, and then cache those results. When verify |
|
108 |
comes looking, look in the cache instead of actually opening the file and |
|
109 |
processing it. |
|
110 |
||
111 |
* Yuya has concerns about when blob fetch takes place vs when revlog is |
|
112 |
verified. Since the visible hash matches the blob content, I don't think |
|
113 |
there's a way to verify the pointer file that's actually stored in the |
|
114 |
filelog (other than basic JSON checks). Full verification requires the |
|
115 |
blob. See |
|
116 |
https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-April/116133.html |
|
117 |
||
118 |
* Opening a corrupt pointer file aborts. It probably shouldn't for verify. |
|
119 |
||
120 |
||
121 |
Future ideas/features/polishing |
|
122 |
------------------------------- |
|
123 |
||
124 |
These aren't in any particular order, and are things that don't have obvious BC |
|
125 |
concerns. |
|
126 |
||
127 |
#. Garbage collection `(issue5790) <https://bz.mercurial-scm.org/show_bug.cgi?id=5790>`_ |
|
128 |
||
129 |
* This gets complicated because of the global cache, which may or may not |
|
130 |
consist of hardlinks to the repo, and may be in use by other repos. (So |
|
131 |
the gc may be pointless.) |
|
132 |
||
133 |
#. `Compress blobs <https://github.com/git-lfs/git-lfs/issues/260>`_ |
|
134 |
||
135 |
* 700MB repo becomes 2.5GB with all lfs blobs |
|
136 |
* What implications are there for filesystem paths that don't indicate |
|
137 |
compression? (i.e. how to share with global cache and other local repos?) |
|
138 |
* Probably needs to be stored under `.hg/store/lfs/zstd`, with a repo |
|
139 |
requirement. |
|
140 |
* Allow tuneable compression type and settings? |
|
141 |
* Support compression over the wire if both sides understand the compression? |
|
142 |
* `debugupgraderepo` to convert? |
|
143 |
* Probably not worth supporting compressed and uncompressed concurrently |
|
144 |
||
145 |
#. Determine things to upload with `readfast() |
|
146 |
<https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-August/121315.html>`_ |
|
147 |
||
148 |
* Significantly faster when pushing an entire large repo to http. |
|
149 |
* Causes test changes to fileset and templates; may need both this and |
|
150 |
current methods of lookup. |
|
151 |
||
152 |
#. Is a command to download everything needed? This would allow copying the |
|
153 |
whole to a portable drive. Currently this can be effected by running |
|
154 |
`hg verify`. |
|
155 |
||
156 |
#. Stop reading in entire file into one buffer when passing through filelog |
|
157 |
interface |
|
158 |
||
159 |
* `Requires major replumbing to core |
|
160 |
<https://www.mercurial-scm.org/wiki/HandlingLargeFiles>`_ |
|
161 |
||
162 |
#. Keep corrupt files around in 'store/lfs/incoming' for forensics? |
|
163 |
||
164 |
* Files should be downloaded to 'incoming', and moved to normal location when |
|
165 |
done. |
|
166 |
||
167 |
#. Client side path enhancements |
|
168 |
||
169 |
* Support paths.default:lfs = ... style paths |
|
170 |
* SSH -> https server inference |
|
171 |
||
172 |
* https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-April/115416.html |
|
173 |
* https://github.com/git-lfs/git-lfs/blob/master/docs/api/server-discovery.md#guessing-the-server |
|
174 |
||
175 |
#. Server enhancements |
|
176 |
||
177 |
* Add support for transfer quotas? |
|
178 |
* Download should be able to send the file in chunks, without reading the |
|
179 |
whole thing into memory |
|
180 |
(https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-March/114584.html) |
|
181 |
* Support for resuming transfers |
|
182 |
||
183 |
#. Handle 3rd party server storage. |
|
184 |
||
185 |
* Teach client to handle lfs `verify` action. This is needed after the |
|
186 |
server instructs the client to upload the file to another server, in order |
|
187 |
to tell the server that the upload completed. |
|
188 |
* Teach the server to send redirects if configured, and process `verify` |
|
189 |
requests. |
|
190 |
||
191 |
#. `Is any hg-git work needed |
|
192 |
<https://groups.google.com/d/msg/hg-git/XYNQuudteeM/ivt8gXoZAAAJ>`_? |