Mercurial > hg
annotate hgext/lfs/TODO.rst @ 51479:5bd31e68c7a3
stream-clone-test: add title to various test cases
These case are fine as is, but as we are adding title to all the other as we
simplify them, lets add title for all cases.
author | Pierre-Yves David <pierre-yves.david@octobus.net> |
---|---|
date | Fri, 08 Mar 2024 10:59:51 +0100 |
parents | 87167caa2989 |
children |
rev | line source |
---|---|
42242 | 1 Prior to removing (EXPERIMENTAL) |
2 -------------------------------- | |
3 | |
4 These things affect UI and/or behavior, and should probably be implemented (or | |
5 ruled out) prior to taking off the experimental shrinkwrap. | |
6 | |
7 #. Finish the `hg convert` story | |
8 | |
9 * Add an argument to accept a rules file to apply during conversion? | |
10 Currently `lfs.track` is the only way to affect the conversion. | |
11 * drop `lfs.track` config settings | |
12 * splice in `.hglfs` file for normal repo -> lfs conversions? | |
13 | |
14 #. Stop uploading blobs when pushing between local repos | |
15 | |
16 * Could probably hardlink directly to the other local repo's store | |
17 * Support inferring `lfs.url` for local push/pull (currently only supports | |
18 http) | |
19 | |
20 #. Stop uploading blobs on strip/amend/histedit/etc. | |
21 | |
22 * This seems to be a side effect of doing it for `hg bundle`, which probably | |
23 makes sense. | |
24 | |
25 #. Handle a server with the extension loaded and a client without the extension | |
26 more gracefully. | |
27 | |
28 * `changegroup3` is still experimental, and not enabled by default. | |
29 * Figure out how to `introduce LFS to the server repo | |
30 <https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-September/122281.html>`_. | |
31 See the TODO in test-lfs-serve.t. | |
32 | |
33 #. Remove `lfs.retry` hack in client? This came from FB, but it's not clear why | |
34 it is/was needed. | |
35 | |
36 #. `hg export` currently writes out the LFS blob. Should it write the pointer | |
37 instead? | |
38 | |
39 * `hg diff` is similar, and probably shouldn't see the pointer file | |
40 | |
41 #. Show to-be-applied rules with `hg files -r 'wdir()' 'set:lfs()'` | |
42 | |
43 * `debugignore` can show file + line number, so a dedicated command could be | |
44 useful too. | |
45 | |
46 #. Filesets, revsets and templates | |
47 | |
48 * A dedicated revset should be faster than `'file(set:lfs())'` | |
49 * Attach `{lfsoid}` and `{lfspointer}` to `general keywords | |
50 <https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-January/110251.html>`_, | |
51 IFF the file is a blob | |
52 * Drop existing items that would be redundant with general support | |
53 | |
54 #. Can `grep` avoid downloading most things? | |
55 | |
56 * Add a command option to skip LFS blobs? | |
57 | |
58 #. Add a flag that's visible in `hg files -v` to indicate external storage? | |
59 | |
60 #. Server side issues | |
61 | |
62 * Check for local disk space before allowing upload. (I've got a patch for | |
63 this.) | |
64 * Make sure the http codes used are appropriate. | |
65 * `Why is copying the Authorization header into the JSON payload necessary | |
66 <https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-April/116230.html>`_? | |
67 * `LFS-Authenticate` header support in client and server(?) | |
68 | |
69 #. Add locks on cache and blob store | |
70 | |
71 * This is complicated with a global store, and multiple potentially unrelated | |
72 local repositories that reference the same blob. | |
73 * Alternately, maybe just handle collisions when trying to create the same | |
74 blob in the store somehow. | |
75 | |
76 #. Are proper file sizes reported in `debugupgraderepo`? | |
77 | |
78 #. Finish prefetching files | |
79 | |
42479
ed9a99564051
lfs: correct an error in the TODO file
Matt Harbison <matt_harbison@yahoo.com>
parents:
42242
diff
changeset
|
80 * `-T {data}` (other than cat?) |
42242 | 81 * `verify` |
82 * `grep` | |
83 | |
84 #. Output cleanup | |
85 | |
86 * Can we print the url when connecting to the blobstore? (A sudden | |
87 connection refused after pulling commits looks confusing.) Problem is, | |
88 'pushing to main url' is printed, and then lfs wants to upload before going | |
89 back to the main repo transfer, so then *that* could be confusing with | |
90 extra output. (This is kinda improved with 380f5131ee7b and 9f78d10742af.) | |
91 | |
92 * Add more progress indicators? Uploading a large repo looks idle for a long | |
93 time while it scans for blobs in each outgoing revision. | |
94 | |
95 * Print filenames instead of hashes in error messages | |
96 | |
97 * subrepo aware paths, where necessary | |
98 | |
99 * Is existing output at the right status/note/debug level? | |
100 | |
101 #. Can `verify` be done without downloading everything? | |
102 | |
103 * If we know that we are talking to an hg server, we can leverage the fact | |
104 that it validates in the Batch API portion, and skip d/l altogether. OTOH, | |
105 maybe we should download the files unconditionally for forensics. The | |
106 alternative is to define a custom transfer handler that definitively | |
107 verifies without transferring, and then cache those results. When verify | |
108 comes looking, look in the cache instead of actually opening the file and | |
109 processing it. | |
110 | |
111 * Yuya has concerns about when blob fetch takes place vs when revlog is | |
112 verified. Since the visible hash matches the blob content, I don't think | |
113 there's a way to verify the pointer file that's actually stored in the | |
114 filelog (other than basic JSON checks). Full verification requires the | |
115 blob. See | |
116 https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-April/116133.html | |
117 | |
118 * Opening a corrupt pointer file aborts. It probably shouldn't for verify. | |
119 | |
120 | |
121 Future ideas/features/polishing | |
122 ------------------------------- | |
123 | |
124 These aren't in any particular order, and are things that don't have obvious BC | |
125 concerns. | |
126 | |
127 #. Garbage collection `(issue5790) <https://bz.mercurial-scm.org/show_bug.cgi?id=5790>`_ | |
128 | |
129 * This gets complicated because of the global cache, which may or may not | |
130 consist of hardlinks to the repo, and may be in use by other repos. (So | |
131 the gc may be pointless.) | |
132 | |
133 #. `Compress blobs <https://github.com/git-lfs/git-lfs/issues/260>`_ | |
134 | |
135 * 700MB repo becomes 2.5GB with all lfs blobs | |
136 * What implications are there for filesystem paths that don't indicate | |
137 compression? (i.e. how to share with global cache and other local repos?) | |
138 * Probably needs to be stored under `.hg/store/lfs/zstd`, with a repo | |
139 requirement. | |
140 * Allow tuneable compression type and settings? | |
141 * Support compression over the wire if both sides understand the compression? | |
142 * `debugupgraderepo` to convert? | |
143 * Probably not worth supporting compressed and uncompressed concurrently | |
144 | |
145 #. Determine things to upload with `readfast() | |
146 <https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-August/121315.html>`_ | |
147 | |
148 * Significantly faster when pushing an entire large repo to http. | |
149 * Causes test changes to fileset and templates; may need both this and | |
150 current methods of lookup. | |
151 | |
152 #. Is a command to download everything needed? This would allow copying the | |
153 whole to a portable drive. Currently this can be effected by running | |
154 `hg verify`. | |
155 | |
156 #. Stop reading in entire file into one buffer when passing through filelog | |
157 interface | |
158 | |
159 * `Requires major replumbing to core | |
160 <https://www.mercurial-scm.org/wiki/HandlingLargeFiles>`_ | |
161 | |
162 #. Keep corrupt files around in 'store/lfs/incoming' for forensics? | |
163 | |
164 * Files should be downloaded to 'incoming', and moved to normal location when | |
165 done. | |
166 | |
167 #. Client side path enhancements | |
168 | |
169 * Support paths.default:lfs = ... style paths | |
170 * SSH -> https server inference | |
171 | |
172 * https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-April/115416.html | |
173 * https://github.com/git-lfs/git-lfs/blob/master/docs/api/server-discovery.md#guessing-the-server | |
174 | |
175 #. Server enhancements | |
176 | |
177 * Add support for transfer quotas? | |
178 * Download should be able to send the file in chunks, without reading the | |
179 whole thing into memory | |
180 (https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-March/114584.html) | |
181 * Support for resuming transfers | |
182 | |
183 #. Handle 3rd party server storage. | |
184 | |
185 * Teach client to handle lfs `verify` action. This is needed after the | |
186 server instructs the client to upload the file to another server, in order | |
187 to tell the server that the upload completed. | |
188 * Teach the server to send redirects if configured, and process `verify` | |
189 requests. | |
190 | |
191 #. `Is any hg-git work needed | |
192 <https://groups.google.com/d/msg/hg-git/XYNQuudteeM/ivt8gXoZAAAJ>`_? |