Just had a quick browse through https://github.com/tus/tus-node-server/tree/master/lib and it looks like it would be fairly easy to rip out the relevant parts and port them to https://github.com/remotestorage/armadietto.
The caveat being that you’d most likely not want to allow large files with the Redis store, but only the filesystem one.
/cc @les
I see, though I’m not sure how that makes us compatible with Tus, if a client that expects a Tus-compatible server to start a multi-part upload would initialize the file a different way then what the rs-server is expecting.
I see the confusion, but if you consider how both Tus and RS work, then it should be fairly clear why anyone can create uploads any way they want, which is why Creation is an optional extension in Tus.
The entire Tus protocol is an extension to whatever server and client people are running. A Tus client cannot be compatible with RS out of the box in any case, because it doesn’t do any of the other required RS client functionality, like connecting an account, getting a token, and using that token as a header to authorize the upload requests. Tus is an extension that people can add to their client/server solutions in order to facilitate resumable uploads in addition to whatever other functionality they have.
Yeah, that makes sense. So essentially we’d be leaving as much RS code in place (creating or deleting files) while adding just the resume/multi-part upload functionality.
However that still begs the question of checksum and expiration/termination extension headers. To me those seem like features we’d really want to have from the get-go to failsafe against fragile network conditions. How else would we either verify a transfer wasn’t corrupted along the way, or abort a transfer (with the resources being removed - I guess that ties back to @galfert’s question as to where the file lives before it’s completed, how do we reference it if not via. the expiration/termination extensions)?
I think there’s a reason those are also optional. They are certainly nice to have, but I’d say it’s arguable if they should be required, as they are not strictly necessary to facilitate uploads.
This is mostly mitigated by knowing the exact length of bytes for the final upload. It’s not bulletproof, but can be considered good enough for the minimum required verification imo.
The expiration extension is obviously nice to have, but a file would just be incomplete until it’s deleted, when that’s not implemented. The server can also prune after a while, regardless of an extension spec. I think the idea of the expiration extension is that if a server has heavy load, and especially public uploads enabled, then the client can know when to give up on a slow, ongoing upload. But without the extension, you would just receive back a upload byte offset of 0 for the next PATCH in case the partial file has been pruned. Which is surely an edge case until RS is a widely used standard and clients attempt monster uploads of massive files in the first place.
The termination extension is already exactly the same as what RS would do, namely a DELETE request to the resource, which should just remove whatever is there–complete or not-- by simple HTTP semantics and existing RS protocol spec. So any RS server already supports that extension by design.
P.S.: I think we agree that servers should probably be allowed to support some of the extensions, especially checksums and expiration.
Just thinking about this some more - so are you proposing we have the full capabilities in the RS client, but have it check for certain features on the server end? In which case, then I agree, I’d just want to use a server that supports them because I can’t think of many downsides to having re-transmission capabilities but can think of plenty of downsides to not having that. I am just imagining most hackerbeaches where internet is spotty and unreliable, we’d want to make sure an upload can easily resend corrupt payloads, etc. The downside being implementation in server implementations, however Tus seems to have most popular languages covered
Yes, the general idea for this has always been that it’s an optional RS extension that is announced in Webfinger, same as for example content ranges for GET requests. And the reference client should of course use it out of the box for any server that supports it.
Someone published a new draft spec for partial uploads, which also uses the PATCH verb:
https://tools.ietf.org/html/draft-wright-http-partial-upload-01
It’s an early draft, but maybe we can consider how it fits into our conversation and considerations so far.
(/ht @catwell)
Looks like perhaps waiting this one out was worth it. There’s this spec draft now, which is also linked from W3C Linked Web Storage developments: use cases