I was imagining a future where remoteStorage (and maybe also Sockethub) is an option wherever Nextcloud, Wordpress, or even Email is a ‘one-click install’. I consider these a kind of ‘app store’ for developers like me who are not well-versed in server management but can use control panels and maybe run a few commands in the terminal; I think there’s a lot of us out there.
From my experience explaining remoteStorage to people, the biggest barrier to adoption has been the lack of providers. I was also told once something to the effect of “it’s hard to see how remoteStorage advances beyond a spec until other people can install it themselves.”
If there are specific barriers to making this as simple as, let’s say, Wordpress (one-click, app store style), maybe let’s enumerate and discuss them here. Would it be easier to realize this kind of project with funding? Who from previous remoteStorage contributors or from the Internet at large might be able to work on this?
Infrastructure is not my strength but I could contribute by writing guides and maybe doing video tutorials on how to set up on those platforms. And I would like to run my own instance so I could also test.
I’d love to take something like Armadietto to a production ready piece of software. I think the minimum functionality required to do this is:
Logging
Docker image so a user doesn’t need to care about node/version requirements and install/setup becomes a single command something like docker run -d --volume "$(pwd)"/storage:/armadietto-storage:rw remotestorage/armadietto
That is enough to get us to the place where a user can easily spin up a local version of Armadietto on a VPS that they own. They would still need to configure a proxy/SSL (nginx+letsencrypt for example), but that is standard for anything that they’d deploy on their server. We go so far as to provide a custom docker image that included nginx and letsencrypt as well so that everything was self-contained.
To enable allowing users to provide easy storage as a service, I think it would be better provided as another project, but maybe just in terms of code-org and not an entirely separate installable. At a minimum that project would need to provide
User management (CRUD for users)
Per user limitations (allowing things like 100mb/user space allocation for example)
Usage analytics
I’m currently working on getting the logging piece sorted out and then I’ll be tackling the docker task. Once those are in I’ll be doing some more testing on the Digital Ocean storage engine on my server. I would like to get us to the point where running a server for a single user is easy and not really any kind of special event. It should just work.
I would add some of the self-hosting solutions, which are already used by eager adopters of decentralized solutions, e.g. nextcloud.com, yunohost.org and freedombox.org.
The main question for self-hosting is of course which server implementation to use. And if there might be a dependency on Apache, which still breaks RS servers. Also, if there’s a central user management system, then it would be useful to have the auth be pluggable in the RS server that’s being used, of course.
True! I’m excited about renewed activity around Armadietto in this regard! Considering that JS is basically the most widely used language atm, it will probably end up being the most active server project.
But also php-remote-storage is still the most mature solution I think, and should be especially easy to deploy when the latest Apache release (which contains the important bugfix for RS) is available in package managers. Furthermore, considering the ever-growing popularity of Rust, we do still have mysteryshack available, which is also a working implementation, and could be picked up by a new maintainer at some point, too.
In general, however, I think it’s still difficult to plan for mass adoption, because the protocol is still an Internet Draft at IETF. So a company-driven solution like e.g. nextCloud is not a good target yet, because the company would have to support potential radical changes to the RS integration, which could happen anytime until the spec is an RFC.
P.S.: People can also still be paid by Digital Ocean for writing tutorials on setting up server software. Would obviously be useful to have some guides for RS servers over there…
This sounds like a great tractable goal—I would run that command . Especially if the PaaS has some kind of zero-config SSL that can be applied to the corresponding domain.
@raucao had mentioned to me similar concerns about needing to be ‘ready’ before pushed out. I think even if it’s crude and one person can setup an account themselves, it’s a huge step towards enabling experimentation without friction. The nature of remoteStorage itself already means that the data is portable, and so if something goes wrong, it’s easy to spin up another instance and rs-backup/restore. Personally, I would prefer experimenting now with something basic and functional to waiting a year or more for something polished.
I updated the list. Great to know about these options, never heard of the last two. I have seen Nextcloud as something you can install but I don’t understand how you install things inside of it—or it needs to be a ‘Nextcloud app’ maybe?
Could we just use a fork and hotfix until Apache fixes that?
I think the idea I heard about ‘remoteStorage not advancing beyond a spec’ is referring to this in the inverse, that it needs to be easily installable to exit draft status in the practical sense: rather than waiting for the spec to encourage mass adoption, maximize adoption until its a spec. But also, if the spec is versioned, do companies need to support new changes quickly? Can’t they pin themselves to an older version if necessary? Perhaps I misunderstood how that works.
My goal with this conversation is to encourage embracing the messy nature of this rather than waiting until it’s ‘ready’, so we can run farther with it and open doors for more people to come in. IMO it feels great as it is, at least with the 5apps server implementation, so it hopefully can only get better from here.
Oh also I’m not sure if this has been discussed somewhere but perhaps due to starting so many years ago, there are ideas of the server implementation implying ‘for a provider to run and host multiple people’. Maybe now that single-tenant architectures are so common, the servers can simplify their implementation to be functional with just one account. Imagine not having to design a login/account/quota system… Obviously a multi-tenant server uses resources more efficiently, but it may not be necessary at the start. There will always be more bloggers than mastodon instance admins.
The spec drafts expire every 6 months. With the spec drafts being published as IETF Internet Drafts, it is generally discouraged to even run RS in production at scale for any company, even though at the same time it can only become an RFC, when there are multiple implementations available. The point is that drafts are not considered stable at all, and an expired draft is not really the same as an older spec version.
So yes, if e.g. nextCloud were to officially support RS before it’s an RFC, then they would have to implement the final RFC while still supporting expired draft versions. As there’s no telling what would be changed in the spec once we hand it to the reviewers, there could be critical, potentially breaking or incompatible changes that would even make this impossible. So I think it is critical that we push the RFC process forward asap if we want mass adoption.
You can find more information about both the IETF process, as well as our discussion around “finishing” the spec in this topic: The road to RFC. Or elsewhere?
TL;DR of the latest status is that we still wanted support for resumable uploads, because single HTTP PUTs are just not enough to upload large images, longer audio, or video files, especially on mobile connections. That discussion is always open, and also the topic of resumable uploads can be picked up by anyone who wants to prototype that functionality (for example with rs.js and armadietto), so we can find out what to spec exactly. The leg work around how this could work has already been done!
Well I am not a programmer, I think it would be nice to have an easily deployable micro service that could run Remote storage spec. I would be willing to be the beta test, write documentation etc. I’m just not a programmer. I can also do a security audit of the design concepts since I am a network security professional by trade.
@symgryph Welcome to the forum, and thanks for responding! That’s great. In my opinion, the more ‘not a programmer’ people there are in the community, the better
@raucau perhaps someone working with the core can make a list of important areas to do a security audit? Version 2.0 could probably use a thorough poking around even if it’s mainly converting to TypeScript. I’m not sure if there has been anyone focused on security since the inception of the project.
Have there been any thoughts to running the demon in a compiled language? This would make lightweight implementations Much more feasible. Node has a huge overhead, I am wanting to run these on small local networks think raspberry pi zero. Having some thing in a very small footprint would be exceptionally cool. Languages that I’ve seen used include: C itself, NIM, go and rust. I looked everywhere for reasonable implementation of a small binary and could not find a single one. PHP requires a very large run time as python etc.
I would be happy to be of help in both architectural and some reviews of the program. I do not directly do source code editing however I certainly can look at things and see if the design pattern is sound.
NIm, go, and rust are memory safe and very perfomant. In terms of member utilization either nim or rust are winners here. Well golang performs well it’s garbage collector tends to introduce lots of variability in performance.
Once the initial changes are in to Armadietto, my hope is to turn my attention to mysteryshack and get that humming along as well. The more options we have for people to use the better.
. Especially if the PaaS has some kind of zero-config SSL that can be applied to the corresponding domain.