State of the Project

[johannesjo]

Hi there!

I am the developer of the open source app Super Productivity.

First of all thanks for this piece of software and most of all the super cool idea behind it. This really should gain more attention and the idea behind it is much easier to understand and appealing than the very technical stuff SOLID does.

Have you considered advertising this via reddit or dev.to by writing an introduction article about? I think many people would love it and given that there are a growing number of idle freelancing programmers in Europe, there might be new capable people willing to contribute.

I have this project on my radar for quite some while now since I am also using google drive for syncing user data in the Super Productivity app. What has turned me off so far, was that I was unable to make any of them work with the remote storage integration. Together with only moderate commit activity, I got the feeling that this might be slowly dying, which would be a shame.

So I am wondering. What’s the state of things? Are there any big plans or is this mostly in maintenance mode?

[raucao]

Welcome to the community!

Super Productivity looks, well, super nice!

As one of the maintainers of remoteStorage.js, I think I can answer your questions.

@johannesjo

Have you considered advertising this via reddit or dev.to by writing an introduction article about?

Every now and then, someone submits RS, or a remoteStorage.js release, to Hacker News, Lobsters, and such. Usually, there are some good discussions underneath, and more people have it on their radar afterwards.

I had planned to write a longer-form post about the overall status of the protocol, including a request for contributions, but procrastination is a monster sometimes. I hope to get that done soon. (Also, @michielbdejong asked me privately to join as a spec editor, because he is working on SOLID now. So I didn’t want to write some big post until that is official.)

@johannesjo

Together with only moderate commit activity, I got the feeling that this might be slowly dying, which would be a shame.

So I am wondering. What’s the state of things? Are there any big plans or is this mostly in maintenance mode?

remoteStorage.js is actively maintained for critical bugs. And from time to time there are even new contributors showing up randomly and helping out, when they fix something not just for their own code, but sharing it with the upstream repos.

The decline in commit activity can be attributed to the library being pretty stable as is, and nobody adding new features at the moment. However, we did actually publish a release of the connect widget add-on, with quite some improvements recently for example.

The next big thing some of us want to add is support for partial/resumable uploads, which has to be prototyped both in a client and a server, as it would be a new spec addition.

Personally, I’m also planning to finish my TypeScript branch and propose switching the codebase to that. I think this will make contributions much easier, by making the code easier to understand and reason about, especially for the sync- and caching-related parts.

I hope this helped with answering the immediate questions. And maybe someone sees this and wants to help out with any of the things I just mentioned? I’d be happy to assist in any way I can.

[johannesjo]

Thanks for getting back to me so quickly and answering all my questions! TypeScript support would be great and I agree with all the points you’re making about. Let me know if I can support you with it. I got some experience with migrating stuff to it even though I don’t know your codebase very well.

I also added writing a post to dev.to to my in case I have time to spare list

[raucao]

@johannesjo

Let me know if I can support you with it. I got some experience with migrating stuff to it even though I don’t know your codebase very well.

Yes, any help from someone with more experience in it would be much appreciated! How about I open a draft PR on GitHub soon, so we can collaborate there?

@johannesjo

I also added writing a post to dev.to to my in case I have time to spare list

That would be fantastic as well!

[johannesjo]

Yes, any help from someone with more experience in it would be much appreciated! How about I open a draft PR on GitHub soon, so we can collaborate there?

Good Plan! We probably should take good care not to work on it at the same time, as there are so many potential conflicts. Maybe we can do it like this: Once we start working we write a comment on the Draft PR and once we’re done (no unpushed local changes) we write another one?

It’s great that you got so many unit tests in place. That should make things a lot easier.

[raucao]

Ready for action:

Convert source code to TypeScript

[muety]

Hi guys! I found out about remoteStorage quite some time ago and since had it “in my radar” as you call it Now I thought about writing my own to-do app (I know, there is an infinite amount of such already on GitHub, but none of those perfectly fits my needs) and I consider implementing it with remoteStorage for sync. Therefore I was wondering what the state of the project is right now and especially if there is going to be (a) a Java / Android client library and (b) more options for (actively maintained) servers some time?

Generally speaking, I really like the idea and concepts of remoteStorage and it is such a pity that the project doesn’t get a lot more attention on the web! Many apps could easily be done with just remoteStorage as a “backend”.

@johannesjo Cool project, by the way (Super Productivity)! It’s almost exactly what I was looking for and, if developed a little further, might actually replace Microsoft ToDo and Toggl for me

[johannesjo]

Cool project, by the way (Super Productivity)! It’s almost exactly what I was looking for and, if developed a little further, might actually replace Microsoft ToDo and Toggl for me

That would be great If you have any suggestions what needs to be changed up, please let me know!

[raucao]

@muety

especially if there is going to be (a) a Java / Android client library

I don’t know of anyone working on that at this moment. Our JS client supports (and is tested with) Cordova at least, so it’s not impossible to create app packages for iOS and Android. I know that’s not what you’re asking, but just adding that for whoever else may be wondering.

Personally, I think it’d be pretty cool to have a Kotlin client library and to be able to add RS to native Android apps. One problem we still have to solve to make something like that more compatible is to separate data schemas/formats from the JavaScript modules.

@muety

more options for (actively maintained) servers some time

I think for people to be motivated to work more on servers, we probably need 1) more apps and thus users, and 2) a stable protocol, instead of drafts that get updated twice a year. There’s a bit of a chicken-and-egg problem there, which is difficult to solve: if you don’t have apps you don’t have users, but without users you don’t need servers. But without servers…

However, there are at least 3 server implementations, which should work out of the box with all halfway modern RS apps (i.e. not very old ones from the early days of the protocol, that haven’t been upgraded). I know, two of them are marked experimental, but that’s also partly because the protocol isn’t officially stable. I definitely know that any help with the node.js one is much appreciated.

Another thing to consider is that once the protocol is stable, it makes much more sense for developers to add RS to existing FOSS cloud solutions, like e.g. NextCloud. They could even expose their existing data to RS apps via adapters.

Yet another consideration is that since the beginning, and I think it’s still the case, there’s a glaring bug in Apache, which basically breaks remoteStorage, by not returning CORS headers for 304 responses, which are an essential part of how RS clients sync data. I think @rosano mentioned somewhere that it looks like it’s going to be fixed soon. I will personally invite anyone who wants to join for a keg of beer, when that finally happens.

[raucao]

@muety

Now I thought about writing my own to-do app (I know, there is an infinite amount of such already on GitHub, but none of those perfectly fits my needs) and I consider implementing it with remoteStorage for sync

I thought about the same thing. I think todo apps are a perfect case for RS, because they could share the same data modules/formats, so you can manage your to-dos from multiple different apps. You could have separate ones where each is great at one context: e.g. goal planning/tracking vs. calendar vs. time tracking, and so on.

I think there’s currently no RS to-do app except for some experiments, but I’d be happy to help with the data module and schema design.

[michielbdejong]

Hi all, sorry for the late reply, just wanted to say hi on this thread as well!

Had a nice and exciting video call with @raucao last week.

The remoteStorage project is now almost 10 years old and I haven’t been very involved in the second half of that, but would like to start making some time for that again!

I would like to eventually step in to coding for remoteStorage.js and related projects again, and @raucao and I also discussed how cool it would be if we could submit our remoteStorage spec internet draft for the track to become an RFC.

Maybe we could do a video call next week or so, just to say hi to each other and see who is currently interested and available to help move the remoteStorage project further along?

[raucao]

@michielbdejong

Maybe we could do a video call next week or so, just to say hi to each other and see who is currently interested and available to help move the remoteStorage project further along?

Public RS Planning/Community Call (April 21 or 23)

[raucao]

@raucao

Yet another consideration is that since the beginning, and I think it’s still the case, there’s a glaring bug in Apache, which basically breaks remoteStorage, by not returning CORS headers for 304 responses, which are an essential part of how RS clients sync data. I think @rosano mentioned somewhere that it looks like it’s going to be fixed soon.

I had another look at this. But I couldn’t find any news really.

• The Apache bug is still open.
• There’s a (closed) issue for php-remote-storage, which is where I saw @rosano’s comment.

Edit: There’s another Apache bug open, with a bit more discussion. The explanation for how the bug came to be is provided Roy Fielding (one of the original authors of the HTTP spec):

The original code that I wrote (in 1997) for the canned response generator (ap_send_error_response) was moved in 2000 to the generic http filter in a mistaken attempt to solve an unrelated bug in 2.0a streams. […] It has no business being applied to all 304 responses.

I think the best solution right now is to simply remove the entire conditional that attempts to reduce 304 responses. If that results in too much data being sent, we should have a configurable list of fields to exclude on 304 responses rather than a fixed table in the source code.

And that list of fields/headers is now being discussed here:

Updating stored headers

And the Apache fix is now dependent on that discussion being concluded.

[raucao]

FYI: Just added a section to the wiki about this, so people hopefully don’t run into it by surprise anymore.

By the way, the wiki is also a wide open, green field for contributing!

[raucao]

@raucao

And the Apache fix is now dependent on that discussion being concluded.

Just FYI: the HTTP spec issue has been resolved, so the Apache fix will be in soon!