Announcing Mercato: A privacy first hosting service for apps in the RemoteStorage ecosystem

App announcements
1

So! I built an app store for web apps using RemoteStorage.

You can check out the GitHub page if you want to skip right to the code

Let’s talk about why I think this is a good idea :slight_smile:

Why build Mercato?

The current ecosystem of apps on the web and phone generally requires too much trust on the part of users. There are lots of stories of simple phone apps selling users’ data and I think it’s time to stop privacy leaks in a guaranteed way, and I believe we can do that while still retaining the usefulness of apps.

I think a simple solution is to stop assuming apps should have network access. A phone app with access to my camera and microphone but no way to exfiltrate any data out of a phone is not a privacy problem. An app can’t violate my privacy if it can’t send data anywhere

My hypothesis is that RS apps tend to be ones where users create content (that’s why owning your data is important for those apps), and that those types of apps are also less reliant on network access. Looking at many of the types of apps on the RS app list - productivity, self-improvement, finance, password - we see that for these types of apps, the core functionality is not about accessing data over the internet. There will always be other types of RS apps that need network access, but I think Mercato can help a lot with the apps where privacy is the most important.

Given this, we should be able to restrict RS apps to only being able to use the network to talk to your RS server, your data will still be synced and your app will still be useful, but the privacy of your data is no longer about trusting the individual app developers.

What’s Mercato?

Mercato is a web based app platform (similar to an app store) that guarantees your privacy by running your apps inside of a privacy sandbox - the original app developers cannot see your data because it is all stored/synced in RemoteStorage and never goes through a server that the developer controls. The privacy sandbox enforces its restrictions by using modern web standards (specifically Content Security Policy).

Using an App on Mercato

Let’s say you hear from a friend about a diary app that’s hosted on the Mercato platform.

  1. Your friend sends you a link to the app: https://diary.nooq.run (right now this link doesn’t work - I’m planning to host a Mercato instance at nooq.run)
  2. You’ll be redirected to log into the mercato instance (eg nooq.run/login) - mercato will request access to the Mercato scope. This has two purposes: 1) mercato client stores a token so mercato can identify you on other devices/if you clear your cookies 2) Mercato uses this to know what RS server you’re using, so when it locks down network access, it will still allow access to your RS server
  3. You can now go to you app’s page (diary.nooq.run) and use the app like a normal RS app. You’ll see RS’s permission prompts for that app like normal.

If a user wants to see all the apps hosted on Mercato, there’s a page where you can browse all the apps (eg nooq.run/listing)

How Mercato works

Basically, Mercato is a static file hosting service that uses Content-Security-Policy to tell your browser to not let the JS code of the app talk to any servers except mercato and the user’s RS server.

My general goals for Mercato are:

  • The Mercato server should not need access to your RS storage (the client does, but it never sends RS tokens to the server)
  • You should not have to change your app to host it on Mercato. In theory that’s true but I want to port over/write some real apps before I say that’s true.
  • Maintain the existing RS permissions model - apps should each have their own unique domain

If Mercato were to become popular, I hope there would be multiple Mercato services being run, allowing for some competition in app store requirements/% of sales taken by the server/etc.

More details on this at Mercato README - how it works - let me know what questions you have!

Tradeoffs

Mercato doesn’t remove the need for all trust: you do still need to trust the folks running Mercato server. If you really wanted to be careful, you could write a browser extension that watched network traffic to verify that. The folks running your Mercato server are accountable to you for maintaining your privacy, just like the folks running your RS server are accountable to you for keeping control of your data.

A lack of network access does limit significantly the type of apps that will work, but I think it works well for many of the knowledge creation apps where users really care about their data (and thus RS is appealing)

Building an app

To build an app, you just write a normal RS app in a SPA style - then you submit it to the Mercato server and it’ll be made available to users. Right now this is a manual process where you email the app to me :slight_smile:

More details on this at Mercato README - developing for Mercato - let me know what questions you have!

Getting Started/Next Steps

  • You can find the code up at GitHub
  • I don’t currently have a publicly available server - I’ll see if I can get a toy one set up today, but I’ll definitely get a reliable one up early next week.
  • Apps: Right now there aren’t any :slight_smile: I’d like to port over Inspektor since I want to use it for developing locally

How can others get involved?

If you’re interested in working on Mercato, let me know! I’m starting a new job soon, so I don’t know how much time I’ll have for building new features/etc, but if it gains some traction I can guarantee I’ll either be spending time on it to ensure it keeps moving forward or I’ll support an official fork to a different maintainer.

Charging for apps/paying developers

I had originally envisioned this as being an actual app store (similar to phone apps stores) where developers could get paid for creating quality apps that respect their privacy. Running an app store is not something that fits with my current work arrangement :), so I figured I’d just release the app hosting part of it. If someone else wants to run one, please do!

Last thoughts

The name Mercato is from the Italian for Market. I don’t speak Italian, so let me know if you’re aware that it has some terrible slang meaning. I enjoy the Italian language theme of the RS community naming (perhaps just the JS components?)

Perhaps there’s some huge hole in the whole premise of using Content-Security-Policy for this? I don’t think so, but if you have any thoughts on the matter I’d love to hear from you. Some real pen-testing would definitely be appropriate

It seems like I’m not the only one hoping to see RS proliferate and that has ideas for how to encourage that. I’m excited to see where all this goes and if we can get traction for RemoteStorage!

Well, that got long! Please let me know of any questions you might have - this is my first attempt to explain the concept as a whole, so please let me know if anything needs further explaining. I may be slow to respond in the next few days, but I’m looking forward to chatting with folks about the idea.

2 Likes
2

Would be nice to try a demo, maybe you could post a link here when you’re ready.

3

Definitely! My family is travelling but it should be ready to try out Monday or Tuesday. I don’t have a very good demo app yet but I’ll have a simple app that lets you see that it doesn’t allow connections out to other servers.

4

okay! It’s ready for folks to take a look.

Nooq is the instance of Mercato that I’m running (Nooq is to Mercato as 5apps is to remoteStorage)

Nooq home page is the main site
Nooq login page is the place to create your account by logging in to rS
Example app is a simple tech demo app that shows that you can successfully connect to rS and that you can’t connect to other urls

5

hey!
I wanted to try out porting apps, so I ported over Inspektor.

I started when I posted my last message (~1 hr ago), and I’m already done.

So I think I can say that porting over is very easy and apps should work as-is :slight_smile:

I originally tried to do a build of inspektor locally, but ran into a bunch of errors during npm install - it looks like it has very old ember dependencies that seem to rely on node-gyp which probably rely on python 2… That looked like a painful mess of upgrading packages. BUT! I didn’t have to worry about that because all I needed was the final built files being sent to users by the hosted app up on the web (inspektor.5apps.com).

So I still want to do more app porting to see what other issues I run into, but in the first test, I passed the test of “Apps should simply work as-is” - in this case, without needing to touch/change any code. You can actually compare the integrity hashes of the script tags in the root html document of inspektor.5apps.com and inspektor.nooq.run and see they’re the same files.

Porting consisted of:

  1. Download all the html/js/css files from inspektor and add them to the mercato apps directory. GH commit - I cheated a bit and copied over the static assets dir from the source code (public/) rather than try and figure out all the files by watching network traffic
  2. Add an entry into mercato’s listing of apps GH commit.

I did need to add a work around for server side routing: eg “inspektor.5apps.com/connect” needs to resolve to the root html file (and then ember uses the url to do its own routing), so I added in the ability to specify URLs and what file to serve for those files.

I’m waiting on DNS/server deploy to propagate, but once that’s done you’ll be able to visit https://inspektor.nooq.run/ and use inspektor

Stephen

1 Like
6

Hey Stephen, welcome to the community!

Mercato sounds interesting. Thanks for posting such a detailed intro and also linking examples now.

That’s not cheating, but a feature of my app repos that is fully intended to be used as such! No need for someone to build from source if they don’t want to deal with the npm dependency hellhole. :wink:

I’m afraid manually adding routes to be resolved is not enough. Many RS apps make full use of client-side routing, and some of mine are certainly using parts of the URL as parameters for document IDs and such.

It’s an app developer’s choice if they do this, of course. But usually, when an app does, then it does so for pretty much all routes being served from that domain or URL path.

7

Hey @trestacos , going to checkout your demos, thanks.

So if I understand correctly, your main mechanism is browser CSP. Mercato is essentially a CDN for static files. Any app from Mercato can be trusted as far as that instance of Mercato appstore can be thrown… what I mean, the trust model of “trust service as far as it can be thrown” is one I like. Is one RS uses. If a RS provider breaches trust, you move your data to another. Same should be true of Mercato providers? As per your description? But that’s a bit tough with an appstore. App stores are naturally hubs and one-offs. Interested to see how this develops. In my opinion, you’re onto something good that fits the RS ecosystem well…

One thing about CSP, RS server dependent, it may not be enough to just allow Mercato’s server and the remote-storage hostname. Sometimes there are other dependencies.

For example, you mentioned you’d want to allow app devs to have the option to be remunerated. There is a RS server option that allows that: rs.overhide.io. But if a Mercato delivered app was being used with this RS server, it’d require CSP to allow stripe.com. The rs.overhide.io server furnishes login widgets that expect access to stripe.com…

But anyways, I’ll play around with Marcato and the overhide.io RS server… might work as is :)…

8

I’m afraid manually adding routes to be resolved is not enough. Many RS apps make full use of client-side routing, and some of mine are certainly using parts of the URL as parameters for document IDs and such.

of course! :man_facepalming: Definitely need to support arbitrary routes. My first thought is that regex matching of paths should support most of those scenarios. Can you think of a scenario that wouldn’t work for that?

If you used more exact route definitions in mercato (eg /posts/.* and /users/.*) you’d have to define your routes twice (once in the app code, once in mercato app definition).

You could also just use a blanket /.* - that’d be pretty easy, but you wouldn’t get 404s. But maybe the JS app’s routing code would catch those scenarios and still show an error.

9

@trestacos , would be interested to see if the last example in this RS tutorial would work with Mercato:

10

So if I understand correctly, your main mechanism is browser CSP. Mercato is essentially a CDN for static files.

Yup, that’s correct. The idea is pretty simple once you realize it’s a possibility, so I really wanted to just go do it and see if anyone was interested in using it.

In my opinion, you’re onto something good that fits the RS ecosystem well…

Thanks! I do agree that the federated model works well and matches RS values.

I do think that app stores naturally gain some user stickiness due to the cost of re-purchasing apps, but I think it’ll be less sticky than phone app stores since RS web apps are very portable and there’s no user lock-in tied to the phone hardware. To avoid lock-in, I’d like to give users a copy of the app code itself and store that in their RS mercato storage. Then build a PWA that loads your apps out of the mercato storage and you’re no longer tied to the app store server. I think at that point, the app store more serves as a trusted listing of apps available for purchase/download. (trusted == the app store won’t inject malicious code into your app)

it may not be enough to just allow Mercato’s server and the remote-storage hostname.

Yeah, I’m very curious to see what exceptions apps might want and definitely think we’ll need some (rs.overhide.io seems like the type we’d want to allow). As a user I’d like to avoid the scenario we have with permissions on phone apps where just to use a simple app users need to approve a bunch of permissions they don’t really want to. Perhaps your mercato’s instance admin could help with determining a whitelist? With mercato instances sharing “good” ones.

I’d definitely be happy to enable rs.overhide.io if you’d like me to. Perhaps you could do some thinking about if an app would be able to abuse overhide to leak data out?

allow app devs to have the option to be remunerated

Yeah, I’d definitely want to enable that. I will admit I’d been more focused on the app store itself mediating the developer payments since that means users don’t have to enter their CC/wallet info on every app, thereby lower the friction for users. Hopefully with federated model of mercato meaning app store “payment tax” would need to be competitive. But I’m also interested to see how web3 payments would work in this model - let me know how it goes with overhide!