Using in Service Worker

I’m adding remote storage to an existing app with a Service Worker.

  1. Are there any guidelines or examples?

  2. Should I instantiate once in the foreground (for remotestorage-widget) and once in the service worker, or proxy the calls from remotestorage-widget via postMessage to the instance in the service worker? For the latter, what calls does the widget make to the remotestorage instance?

  3. When the service worker is running without a foreground webpage, it will need to pass a token to connect(userAddress[, token]). How can I extract the token from the remotestorage instance or remotestorage-widget, so I can store and use it later?

Welcome!

I’m not aware of anyone having tried to use the library in SW before. Also, last time we checked if we may be able to move the sync logic to SW background sync, that feature wasn’t available in browsers yet.

No idea really, but I have personally not seen code using a complex JS library in SW before. If you use caching, the sync is usually pretty fast after re-opening an app, and one can tune that performance a lot by adjusting caching and loading behavior.

The widget (mostly) uses the library’s public API, as documented in JavaScript API — remoteStorage.js 2.0.0-beta.6 documentation.

It’s actually not much code, so if you’re proficient in JavaScript, it should be possible (and maybe interesting and fun) to find out by reading its JS code in remotestorage-widget/src/widget.js at master · remotestorage/remotestorage-widget · GitHub (search for this.rs. to find the calls on the rs.js instance).

The token should be available from remoteStorage.remote.token (source; remote is a WireClient). Or, if the SW has access to the same localStorage, then you could just read it from there directly.

Thanks! The browsers that don’t support ServiceWorkers (https://caniuse.com/#feat=serviceworkers – IE and Opera Mini) don’t support other features my app needs.

The RS docs don’t appear to mention member properties on RS that can be publicly accessed. So far, I’ve found the Widget using apiKeys, remote, hasFeature(), backend and reconnect(). Are they private API that shouldn’t be accessed by anything but the Widget?

Thanks for letting me know about remoteStorage.remote.token – it might be useful to document that.

The widget has been split out of the core library with the 1.0.0 release, and we tried to make it so that it only uses public API, so developers can use the same ones to implement their own UI. However, there were some private API remnants and it seems we have overlooked some of them.

I’m not sure why reconnect() is missing from the docs (it’s the latest addition), and we can probably make some of the properties public as well.

hasFeature() is definitely more of an internal thing, but maybe it makes sense to expose some of them via a public property or function (source)

Currently, wireclient.js uses XmlHttpRequest, which is good for wide compatibility. However, to make it work in a Service Worker, it will need to use the Fetch API. I can start work on that; is there a migration plan? Would a pull request be accepted to change wireclient.js to use the Fetch API if available, with a fallback to XHR?

If there’s a working fallback to XHR, then I don’t see why anyone would object. You could e.g. use the feature list/checks that I linked above for the actual implementation. Those already do fallbacks from IndexedDB to localStorage to memory for example.

I have wireclient updated to use fetch if available. It passes the unit tests and JSHint. However, it’s not clear what other modules will also need updates (discover is likely to).

Are there integration tests, or end-to-end tests I can run? Is testing it with myfavoritedrinks enough?

Discovery only happens when the user is asked to provide their user address, so that shouldn’t happen in a ServiceWorker script. Same for the Webfinger requests.

Testing with myfavoritedrinks should be enough.

That said, I’m still not sure what exactly you made it do in the ServiceWorker. I thought using e.g. sync in the background would necessarily require to use the browser’s background sync API for example. But haven’t really looked into it in detail yet. So it’s not entirely possible to know if testing with myfavoritedrinks is enough, and what to test for exactly in the first place.

I’ve submitted a pull request, which changes wireclient to use fetch() if available: https://github.com/remotestorage/remotestorage.js/pull/1132

1 Like

Is there a guide or example of using remotestorage.js without caching? Obviously, I’ll be using the methods described at https://remotestoragejs.readthedocs.io/en/latest/js-api/base-client.html#data-read-write-operations , but it’s not obvious what order to call them in, when the user may be re-connecting to a server he’s used before, so the items on the server need to be merged with local items, and any conflicts resolved.

Not sure I understand the question. Merging items and handling conflicts is exactly what all the caching/sync code provides.

My app currently stores data in IndexedDB, so it seems redundant to have remotestorage also store a copy locally.

Should I fetch all remote notes to check for changes when remoteStorage emits an “connected” even? A “network-online” event? Other events? Other situations?

Is there a method to generate the Etag for an object?

The idea of remoteStorage.js is to use it like an ODM/ORM, in that you store all documents in its database and let it handle sync via the built-in caching and sync logic. If you want to re-create all of that functionality, then you will most likely want to build your own client instead of using remoteStorage.js in the first place. You could still copy some things like e.g. the discovery and OAuth code, but there’s not much use in loading the entire library and then hack around the fact that it’s built for syncing data between a remote storage and a user’s local IndexedDB.

However, no matter which way you go, you can check exactly how remoteStorage.js handles what you want to do in its source code (namely sync.js, cachinglayer.js, syncedgetputdelete.js and wireclient.js).

Is it possible to build a version of remotestorage without WebFinger and OAuth, for use in a Service Worker or Node? (WebFinger is written to use XHR) Would I need to write a new configuration for Webpack?

We already support node.js out of the box: https://remotestoragejs.readthedocs.io/en/latest/nodejs.html

I don’t see why you would want to use Webfinger in a Service Worker. It’s only needed for when a user personally connects their storage. That is in a normal browser window/tab, not in the background.

Out of the box, remotestorage requires XHR to be implemented, because WebFinger requires it. I’d like to build a version of remotestorage without WebFinger and OAuth, as they can’t be used in a Service Worker and prevent remotestorage from being loaded.

This is not required for use in Node, but would be of value to Node users who prefer to polyfill fetch, rather than XHR.

Considering the content of https://github.com/remotestorage/remotestorage.js/blob/master/src/remotestorage.js I think it would be easier to add fetch support to Webfinger.js than to to rip out Discovery from remoteStorage.js.

It’s a fairly small library, and maintained by @silverbucket (who you know from the resumable upload work): GitHub - silverbucket/webfinger.js: library for easily querying a useraddress for its webfinger record

I don’t think the vast majority of node users really care which HTTP libraries the submodules of their modules are using. It’s required automatically and set as a dependency, so users are only confronted with it when they look into the source code.