Adding new dependencies

remoteStorage.js
1

Continuing the discussion from Setting up a remoteStorage.js development environment:

In the past, we had an unwritten, yet rather clear policy of not adding new dependencies to remoteStorage.js, unless absolutely necessary.

In fact, we’re already planning to remove the biggest one, Bluebird, because not only is Promise a standard now, integrated directly in modern browsers, but also if a developer wants to shim it, they will likely choose a library themselves, that does so across all of their app code anyway, so Bluebird would both add to the size as well as potentially interfere with that. (localForage actually offers seperate builds with and without promise shim for that reason for example).

That said, I personally think it’d be useful to break up remoteStorage.js into smaller components eventually and share more code with other projects.

Now, regarding the SAFE addition in particular (all just my own opinion, not speaking for anyone else here):

  1. We should probably solve the Bluebird issue in the same go, while adding SAFE. At the end of the day, we only need standard promises, without fancy extra features that are not part of the spec. I think any SAFE code should be fine with that as well
  2. The safenet library looks too big and multi-purpose to pull the whole thing into remoteStorage.js from what I’ve seen on first sight.
    • For some things I don’t know what they’re for (DNS features e.g.), for others I think it’s duplicating existing logic (checking for existing auth e.g.)
    • I don’t know why someone would want to encode/decode binary data in/from base64. Might make sense for some apps, but those could pull it in themselves. Maybe there’s a reason for safenet to include it that I can’t see.
    • Using Fetch (via isomorphic-fetch) is a good idea for new libraries, but it’s more of a personal preference than a necessity there. We’re fine with regular old AJAX and as SAFE seems to offer an HTTP API, I think we should use the same mechanism for SAFE requests as for the others. Not withstanding the fact, that I think we should eventually update all code to use Fetch anyway. But it’s a bit early for that imo, as it only works in two browsers as of yet.
    • tweetnacl seems small enough to pull in as a new dependency if needed for SAFE operations. I thought the SAFE Launcher would take care of crypto, so I’m not sure if it is necessary to have for using the API.

In summary, and looking at the SAFE Launcher docs, I think we should not pull in that library, but rather program SAFE support using the launcher API directly, and pulling in new dependencies to remoteStorage.js only if we deem them necessary for implementing SAFE operations. I think the Launcher gateway API looks straight-forward enough to implement it directly in the SAFE backend class.

1 Like
2

Just to note here… yes, that sounds sensible to me so I’ll proceed on this basis, and will ask about the questions raised re tweetnacl and base64.

I wish. It’s all hard for me ATM but worth the learning :slight_smile:

For the Promise stuff, should I use a promise polyfill now? (BTW shim and polyfill are terms I only learned due to this post - to give you an idea :slight_smile: ). I’ll assume yes and use this: GitHub - stefanpenner/es6-promise: A polyfill for ES6-style Promises

3

@raucoa I’m banging my head on a brick wall trying to get the raw MaidSafe example code working just by pasting it into my SAFE backend. I can’t figure out how to do the “require(…)” steps and then access the libraries, as in:

// libsodium wrappers is used for crypto
var libsodium = require('libsodium-wrappers');
var httpRequest = require('request');

// Generate Assymetric Key pairs
var assymetricKeys = libsodium.crypto_box_keypair();
// Generate random Nonce
var nonce = libsodium.randombytes_buf(libsodium.crypto_box_NONCEBYTES);;

…and so on.

I’ve tried searching for other instances of using libraries and inserted require() statements in various places but can’t get the desired effect. Can you give me a simple step by step of what I need to change?

4
  1. That’s node.js code, not browser code.
  2. You don’t need httpRequest, because we use the browser’s own XMLHttpRequest. You can see how the Dropbox backend makes request with it in these lines
  3. I think you don’t need an EMScripten port of libsodium there (that library alone is bigger than all of rs.js). I’m not sure I can help much, as I’m not a crypto expert and have never used it, but it should be possible to use the Web Crypo API for these things
5

We currently don’t use any module loader/transpiler like Webpack or Browserify. So all code is just executed in the browser using global variables. I think we should introduce ES6 modules at some point in the not too distant future though.

6

Thanks.

I’m not intending to keep using these particular dependencies. I was using them in order to start with something that works by using MaidSafe code with minimal changes - then to adapt step by step once I know I can talk to SAFE Launcher.

I have tried global.libsodium = require('libsodium-wrappers'); and a dozen other ways of trying to figure out how to call any of these dependencies, or get them included by the build, but I’ve totally failed.

I can’t get them included, and my attempts to use global like the above says undefined - but I really don’t understand this stuff yet so may be missing something very basic. Anyway, I haven’t figured out how to add in a library and use it so I’m hoping you can explain in simple steps what is required to achieve this. If not I’ll try switching to XHTMLRequest and the WebCryptoAPI.

7

That’s because global is node.js API, not browser. It’s node’s equivalent of window in browsers. The require you see is also node.js, and not implemented in browsers. If you want to use it in a browser library, you need to use a module loader like Webpack or Browserify, which would give you a similar method.

8

But how are you doing it (e.g. for XHTMLRequest, tv4 etc.)? I was trying to follow that for libsodium but couldn’t figure it out.

9

XMLHttpRequest is implemented in Browsers since IE 5. It is what people commonly refer to as AJAX. It’s the browser’s built-in JavaScript API for doing network requests.

tv4 is just loaded as code and then called from the global variable the library instantiates (strictly speaking defined on the browser’s window, but anything defined on window is just available without prefixing it).

10

Thanks for your help @raucao. I have a POST/RESP with SAFE Launcher telling me I’m unauthorized (which is good atm) :slight_smile: For the record I was able to get rs to include libsodium and request from node.js (also had to add bops for node.js Buffer support).

What I did…

cd ~/src/rs.js

npm install -g browserify
bower install libsodium.js

npm install bops request
browserify -r bops -o lib/bops.js
browserify -r request -o lib/request.js

Then add the following paths to components.json in two places as noted above:

bower_components/libsodium.js/dist/browsers/combined/sodium.min.js
lib/bops.js
lib/request.js

In my code I used the Object “libsodium” or “var nacl = libsodium” (for code that was written expecting it as ‘nacl’). And to get the request and buffer objects:

binary = require('bops');
HttpRequest = require('request');

Then…

// Converting assymetric public key to base64 string
// WAS publicKey: Buffer(assymetricKeys.publicKey).toString('base64'),
publicKey: binary.to(assymetricKeys.publicKey,'base64'),
// Converting nonce to base64 string
// WAS: nonce: new Buffer(nonce).toString('base64'),
nonce: binary.to(nonce,'base64'),

etc

Bops is a portability layer for Buffer (a node.js API that is not portable): https://github.com/chrisdickinson/bops

When I have things working we can look at stripping out the extras.

11

Overlooked this one earlier:

You don’t need any other polyfill, because we already include Bluebird.

1 Like