Working on a social library on top of remotestorage.js

ok i’ll add encryption, good point.

Please don’t do that alone. It’s quite a feature in itself and we already have ideas for good UX and implementation. Shouldn’t be too hard, but please instead of just hacking something together yourself, either wait until we or others join your efforts in developing something sustainable with high quality – and most importantly more than 2 eyes from the start – or wait until @silverbucket and I get to it. But from what your planning to get done during the next weeks, I don’t see how this feature could be a part of it (time-wise).

@silverbucket How about we do kind of a crypto hack day somewhere during the next 2 weeks and put our ideas in code?

As a general aside on the topic: the Web Crypto API has just gone last call. Good stuff coming up: http://www.w3.org/blog/news/archives/3755

Good idea. For same service (say Twitter) may exists several libraries with different API and any library should be able to store own data in a secure storage.

So for Twitter we may store several access tokens and use several APIs. It probably makes sense to keep all sensitive data under /user/Secure/* path and access to all such data protect with master password via WebCrypto API.

i just used sjcl, should be high quality afaik

I’m talking about the whole feature and how it’s implemented, not just a crypto lib. Flow, UI/UX, testing, etc.

1 Like

Yeah, my initial thoughts were to use a ‘credentials’ module, but others (I believe @michielbdejong was one of them) said this would be not good enough because then any app that is justified in storing credentials could access other credentials for systems it’s not meant to know about.

I suggested a submodule architecture, so you could do credentials/smtp credentials/twitter, etc. but it’s out of scope for the current roadmap of remoteStorage.

The other option, is to have every type of credential named in it’s own module:

  • credentials-xmpp
  • credentials-smtp
  • credentials-irc

etc. etc.

Ugly, but:

  1. clear to the user
  2. securely partitioned so apps cannot access things they don’t have a reason to

I think a credentials module is a good start, but also think encryption functionality is vital to things like the messages module, which could potentially have years of history of email and chat messages, think gmail history. Something you want to feel comfortable knowing the service administrator can’t read.

Good points!

I’d leave it up for discussion right now and gather more feedback, but which modules the credentials for a certain app or use case are stored in doesn’t impact the general feature development of being able to request or force encryption for a module or data within it. Right?

I figured remoteStorage.social is not a good name, it should at least be unhosted.social, since it depends at least as much on sockethub as on remoteStorage. But then again, calling it unhosted.social would suggest that it is the one unique unhosted social web app tool, so I’m just calling it meute now. So far I implemented:

  • meute.addAccount() for configuring sockethub and irc
  • meute.join() for joining rooms (only irc so far)
  • meute.send() for sending a message (only irc so far)

Its functionality mainly comes from sockethub-client, but with a few additions:

  • compatible with latest sockethub (will upstream this fix to sockethub-client at some point)
  • stores encrypted credentials on remoteStorage
  • takes care of waiting for the registered callback before sending the set verb, so that configuration can be done synchronously from inside the pageload. That makes the API a lot easier to use than sockethub-client’s API (which is quite low-level in that sense).
1 Like

/me edited the thread title from “Working on remoteStorage.social”

Yes, for now I went with that option. I named them just ‘irc’, ‘twitter’, etcetera for now, instead of the full ‘twitter-credentials’. Less accurate and less descriptive, but shorter. I may end up changing it back before submitting these modules upstream to the modules repo, don’t know yet.

1 Like

IMO such short names are better than credentials-irc because module can keep not only credentials but other private data related to the service. For example we can keep in smtp module not only credentials but also last 20 received emails messages.

Actually via RemoteStorage.defineModule we can also define encryption/decryption for sensitive data inside module (see Encryption option in library? - #20 by dogada for more details about this approach).

End even more, actually in such universal modules like ‘smtp’, ‘irc’, ‘twitter’ we can define all their API methods instead of defining API-methods in huge remoteStorage.social or unhosted.social. If I only need twitter and smtp functionality, I will add only these 2 modules and will call directly rs.{tweet,retweet,reply} or ‘rs.smtp.send’. IMO it will lead to more modular approach when for same service can be used different libs by different developers.

BTW, what about using internally alias rs instead of remoteStorage (similar to $ instead of jQuery or _ instead of underscore)?

Exactly! Let’s try to keep things as modular as possible. Also, we need to clearly define boundaries of what remoteStorage modules are supposed to do, and what should go in separate libraries and repos.

Sounds good to me.

ok, but then you will need to add sockethub-client and sjcl as well. i’m not sure if we want to add those two into the standard remoteStorage build? also, adding them to each module separately would make each module usable by itself but would be an ugly solution.

there was a ‘with-modules’ build at some point, and the start of a script where you can build your own custom bundle into one minified file, but i don’t think it ever got much traction.

for now i’m sticking to modules twitter, facebook, irc, email, feeds, and inbox (a unified ActivityStreams inbox with timestamped pointers to items in the other 5 modules), and it is possible to use only the twitter module, but then you will need to include remotestorage.js itself (obviously), and also sockethub-client, and sjcl.

the module will fail to initialize with a useful error message if not all its dependencies are present. any idea how we can do better than that?

This is important:

What you said just now means it is completely undefined and a remoteStorage module can be just about anything. You also didn’t follow up on the other discussion on GitHub, where I proposed to define “functional modules” vs “data modules”.

If it needs SocketHub, it shouldn’t be a remoteStorage.js module in my opinion. It all sounds very much like you want an unhosted social library plus some RS modules for storing a bit of data.

Edit: I also think sending messages and wrappers around that API should be added to the sockethub client or a wrapper of it, not the other way around.

IMO it’s ok to let application developer to include all required for the app functionality javascript libraries. I think good example is approach used by jQuery plugins. When you need $.cookie functionality you need to include jquery.cookie.js plugin in addition to the core jquery.js.

ok, then let’s do it that way for now. we always have the option to change our mind in 6 months from now if we see it’s not working as we had hoped.

yes, so apart from the feeds and articles modules which we got from Dogfeed, the modules i am adding now store credentials encrypted and message history unencrypted.

the modules depend on sjcl but not on sockethub-client. they are called irc, twitter, facebook, email, and inbox.

they are remoteStorage modules in the usual sense, no different from the existing ones. i’m not sure if that makes them functional modules or data modules, then, i’ll find that github issue again and reply there

Much appreciated. Thanks.

I think our final encryption feature should give users the option, yet let the developer “recommend” it by setting a default option.

If you have 100% control over a machine and use disk encryption for it, you might want to disable encryption even for credentials in order to increase convenience. If you have a to-do list for a political revolution in your country, you might want to encrypt that, while most people won’t care about their grocery shopping lists being ultra-secure or not.

Leute, Leute. Ihr seid mir heute aber eine bunte Meute!

ok, made it so that no encryption is used if no password is specified in my strawman implementation https://github.com/remotestorage/modules/blob/functional-modules/utils/credentialsstore.js