remoteStorage

Encryption option in library?

@dogada the file extension is a a possibility, though I’m not sure if that’s a good long-term solution as it could cause issues. Also, we don’t normally store anything with extensions, the modules themselves decide on naming patters, and usually with advanced usage there are more complicated indexing going on to ensure speed and organization of large amounts of data (at least, that’s the goal).

It’s been on my roadmap for the past couple months to do this (implement a way to encrypt data, and retrieve it easily via. the widget), even just to prototype it, but I’ve been pretty busy lately with other things. I hope to get to this soon.

Currently remoteStorage is kind of backed-up with some large changes coming down the pipes, so I’ve been delaying working on this until the dust settles.

If you are interested in helping out, perhaps we could talk this over more (@raucao as well).

As I said, I’d propose to do something like a crypto hack day, where we could remotely pair on prototyping the whole feature in order to see if our ideas would work and gather feedback on it. How about putting up a Doodle for a date?

This is crucial IMO, I’m very interested in getting this done.

How can I help?

Howdy all, I’ve been working on a side project that implements encryption into an unhosted-style library (https://github.com/diafygi/byoFS). It’s still super alpha, but it’s been fun so far. There’s some trade offs in order to provide client-side encryption, and I’m not sure if it would be compatible for the remoteStorage spec.

The biggest tradeoff: I’m only allowing direct key lookups for reads and writes, since the remote server stores only hashed filenames. This prevents listing all the files or searching for a filename in a remote server since the hash is one way. For the byoFS demos, they use a known “index” filename to keep track of all the other filenames, so inserting or changing a filename actually requires updating the index file, too (e.g. two writes on the remote server).

If remoteStorage didn’t want to make that tradeoff, it could keep the filenames unhashed in order to allow listing filenames. I chose not to make that tradeoff and force the app developer to manage their own filenames.

Alternatively, you could encrypt (instead of hash) the filenames and then decrypt when listing files. However, that would mean any filename search beyond a straight key lookup would require downloading the entire list of files and decrypting them all to see if they match the search. Nowhere near ideal, but possible.

2 Likes

For what it’s worth, I also use encrypted credentials storage in https://meute.5apps.com/. It still needs error handling added, but as long as you don’t mistype your master password, it works.

Oh, and my plan is to submit this upstream, of course. See also Working on a social library on top of remotestorage.js

Well, I see 2 main use cases for encryption support:

  1. Provide encrypt/decrypt routines for explicit encryption/decryption.
  2. Allow to to enter master password once and then implicitly decrypt data on read and encrypt them on write.

First use case is simple. We just need to add encryption support based on proven standards (WebCrypto?) to allow application developers to call RemoteStorage.encrypt(data, secret) and RemoteStorage.decrypt(data, secret). To use it developer will only need to implement an UI callback that will ask for password.

Second use case will need some way to detect should an object be encrypted/ decrypted? I agree that extension isn’t good idea and probably better idea is to use RemoteStorage.defineModule. For example if an module defines preSave and ‘postRead’ in own exports RemoteStorage can call these functions immediately after read and before save to the underlying datastorte. Encryption/decryption (and any other pre- and post-processing) can be done inside these functions. Functions accepts raw objects, analyse it, change if required and return original or modified copy. What do you think about this approach?

We also will need a default way to ask user for master password (it can be just trivial window.prompt. Application developer can customize this prompt also on module level defining in exports callback with name like passwordPrompt. Once we obtain master password from user, we can derive real password with a standard function like PKDF2 and use it for actual encryption/decryption with AES256 as chipper. So user will enter master password only once and we will use it until session expire or an timeout. We can discard master password immediately after generating of derived password and keep it private in memory only.

We even can have option to use own master password for each module, this
will lead to better security but weaker UX probably.

Yeah, I’m interested in it. If we all agree desired solution and requirements to it, I can implement it.

developers already have sjcl.encrypt(‘password’, ‘data’) and sjcl.decrypt(‘password’, ‘encrypted-data’) available, so that part is already done, i think? unless we choose to compete with sjcl, but i don’t think that’s a priority at this point.

i agree, the place to enter the password should be inside the app, for two reasons:

  • putting it in the widget, next to where people put in their storage address, suggests that people should put their storage password in there, it will be very confusing i think
  • putting it in the widget suggests that the app has no access to it, and the password is under the control of remotestorage.js. this will not be true though, because an app could easily inspect the DOM

therefore, it is clearer to make it so the user is giving their client-side password to the app, not to the widget.

Cool, great! Can you (and others too, please!) give me some feedback on https://meute.5apps.com/upstream/remotestorage-modules/irc.js maybe? It is my proposal of how we can do this. It embodies the following design decisions:

  • filenames are not encrypted, you can clearly see where the ‘irc-credentials’ crypto-blob lives on the storage.
  • sjcl is used as the crypto library.
  • the user inputs the master password into the app UI, not into the widget.
  • there are no derived passwords, the master passwords stay in memory until you leave the page.
  • the module (in this case the ‘irc’ module) dictates which items should be encrypted and which not.

This last design decision means that all apps using the irc module need to also include sjcl.js, and they also need to make the user enter their master password before the credentials can be retrieved. You can’t have a situation where one app encrypts the irc credentials, and the next one doesn’t.

Please discuss! Once we discuss and reach consensus, we can start implementing.

I disagree with this one, because it means that every password input will look different. The widget is optional anyway (at least it should be), so if you don’t use it you have the form in the app. But if you use it, it should be a unified experience for users and a predefined flow and UI for developers in my opinion.

I think that the remoteStorage library should not depend on the implementation details of SJCL or any other encryption library. We should depend on encryption standards instead.

Native WebCrypto (all major browser vendors working on implementing it ATM) will allow to achieve much better security than any pure javascript approach. We should agree what we will use by default and encrypt/decrypt with best available tool on the platform. Right now it can be SJCL for web application, Mozilla NSS for Firefox extensions or WebCrypto in a year or two.

We should also take into account that data can be encrypted now and decrypted now with SJCL and decrypted in several years with WebCrypto.

We can agree that by default we will use AES-CBS-256 or AES-CTR-256 and this defaults will be forced by remoteStorage.{encrypt,decrypt}. I think we should not recommend to user use directly sjcl.{encrypt,decrypt} because SJCL uses own defaults. For example, by default it uses for encryption AES-CCM-128 that aren’t available in current WebCrypto spec http://www.w3.org/TR/WebCryptoAPI/#algorithms-index.

Advanced users can still use SJCL or any other library directly of course. But with this flexibility they will take also all responsibility to support these encoding schemes during application lifetime.

1 Like

ok, so how can we reach consensus on these two issues? afaics, the option to use WebCrypto is not realistic in the current browser landscape, right? i’m hesitant to try to roll our own crypto lib, i would rather rely on something that comes from Stanford and is generally considered the leading go-to option at current state of the art.

as for adding a password UI to the widget, i think that’s a non-urgent enhancement, but i can see how it would be nice to have. maybe we can do “vote by doing” on that one? the people who want to have it can be the people who build it. is that reasonable?

Nobody said anything else here. I’m not sure you’ve read the last post carefully enough. It raises very good points about using common defaults in our own library and using SJCL as a shim (which could be the only option for now).

strawman iplementation submitted: https://github.com/remotestorage/modules/pull/31

As far as i can tell from http://bitwiseshiftleft.github.io/sjcl/doc/symbols/sjcl.mode.html our only choices are ccm, gcm, and ocb2, no mention of cbs or ctr (I don’t know what those abbreviations mean though).

Let’s just prefix the cyphertext with ‘AES-CCM-128:’, that way we are prepared for switching to another cypher if we ever need/want to remove AES-CCM-128 support.

see https://github.com/remotestorage/modules/commit/aa7af3fca5a465df40330fef7e9adea6f9592ed5 for an example of how we can do this

CCM, GCM and OCB2 are all authenticated encryption modes, CBS and CTR are unauthenticated encryption modes.

SJCL is high-level library and it promotes to use most secure authenticated encryption modes, WebCrypto instead is low-level library and provides low-level primitives which you can use to for authenticated encryption.

ATM WebCrypto mention only one authenticated encryption mode GCM, that also support SJCL AFAIK. So from the portability point of view is better to use AES-128 in GCM mode as default symmetric encryption method IMO.

Please note, WebCrypto API isn’t finished yet and list of supported algorithms may change in nearest 2 month.

Flock implemented a per-file encryption mechanism on top of webdav. I think its key exchange mechanism (if not more) would be a perfect fit for remoteStorage.

1 Like

Any thoughts on the Forge library for crypto?

I’ve come across multiple benchmarks that put Forge significantly ahead of sjcl in terms of encryption and hashing performance. This one, for instance, puts Forge at roughly an order for magnitude faster. It’s rather dated, and there are some curious differences in the encryption modes used between the different libraries, but I think this means maybe we should do some more benchmarking before blindly defaulting to sjcl.

I also happen to prefer Forge’s documentation, as every function comes with multiple examples on how they can be used, compared to sjcl’s which is rather barebones.

Security and correctness is IMO more important than performance, and I guess if SJCL comes from Stanford then you can assume it’s pretty good.

As for the benchmarks, they don’t even let the libraries do the same exact operations, so I’m not sure how to compare them with that.

Also I think we’re mainly interested in the performance of block ciphers (AES) where SJCL (“stanford”) happens to show pretty good performance.

FYI, in case anyone was interested in the flock protocol (which Moxie helped work out, which I feel gives it credbility):

https://github.com/WhisperSystems/Flock/issues/75

There used to be better docs hosted in another repo, but they disappeared. Currently asking about it being rehosted for your benefit :slight_smile:

https://twitter.com/patconnolly/status/697156198248779776

1 Like