i’m developing three constructs which may be of use to other module developers:
PrefixTree:
imagine you want to store 10,000 objects. putting them all in one directory becomes inefficient. PrefixTree creates a small folder tree based on key prefix. For instance, where the PrefixTree has depth 3, the path “tree/foobar” will be stored as “tree/f/o/o/_bar”. the nice thing about PrefixTree is that it’s transparent. you put it in front of a baseClient, and it translates keys to paths and paths back to keys. The depth of the tree is adjusted automatically; whenever a certain leaf becomes too busy, it starts growing subfolders.
SyncedVar:
even when using promises, coding with asynchronous functions can be verbose and cumbersome. If all you want to do is store something and retrieve it, then use a SyncedVar. it has a synchronous set() method and a synchronous get() method. SyncedVar uses asynchronous synchronisation for write, and change events for read. just make sure you wait for the initial sync to be ready before calling get(), or it will throw an exception.
SyncedMap:
same as SyncedVar, but for a map (JavaScript object). it has a synchronous get(key) and a synchronous set(key, val) function. internally it uses a PrefixTree to make sure you can store loads of keys in it without bogging down the sync cycle.
these utils are still under development, i’m using them in the contacts and money modules. I’ll publish all of it when it’s ready.
How’s development progressing?
Have they been already been deployed, though being alpha, to GitHub - remotestorage/modules: DEPRECATED? This’d be good so others could collaborate.
we should definitely collaborate on this, @raucao also just proposed now on irc to do a modules hack day.
the modules i’m trying out are here: https://github.com/michielbdejong/meute/tree/master/remotestorage-modules
an important design choice is that they cache stuff in memory and then expose a synchronous interface.
development is paused because of https://github.com/remotestorage/remotestorage.js/pull/595
I really like the idea of PrefixTree.
As for SyncedVar/Map - I don’t immediately see the point in introducing yet another paradigm for storing/fetching - which is syncronous yet requires registering listeners for events?
Perhaps you could illustrate with an example the problem you see, and an example of how SyncedVar/Map fixes this? I don’t like the idea of adding complexity and confusion which doesn’t really address specific issues that we are suffering from already.
Also I’d like to add that, whatever the use-cases are for SyncedVar/Map I think PrefixTree is URGENTLY needed. @michielbdejong would you be able to focus on it first, instead of developing all three in tanded?
feel free to use PrefixTree in your modules! i just didn’t publish it yet because i first wanted to see if i can get the whole system working, including efficient data import/export and efficient sync with the server. right now, if you use PrefixTree in an app and properly fill it up, it starts to try to do so much sync traffic at once that chances are you’ll crash your tab, and maybe even your whole browser 
at least that’s my experience.
re use case for SyncedVar and SyncedMap - both are just 5 or 10 lines of code i think, so you might as well use the pattern without giving it a name. SyncedMap could become more useful once we batch storeObject calls together into bigger storeObjects calls, then it will add efficiency while writing to IndexedDB.
but mainly, the concept of keeping data in memory is crucial, i think. one could easily make the mistake of using calls to the baseclient in while-you-wait actions, and i think they are too slow for that (especially writes; reads may be ok).
