Creating an image proxy with Meteor — part 1

Avoid insecure warnings when loading external images over https

At Postjar we enable users to save and share web pages within a team. However, those web pages contain images that are sometimes loaded over http, while Postjar is serving https. So when those images show, you get those insecure warnings in the console, and worse, your nice little green lock in the address bar turns grey.

I’m really not an expert on this, and it’s hard to find a decent guide / info on the net how to solve this. My first take was to upload the images to S3 and load them from there, but that required 3 round trips. And in the meantime the user wouldn’t see an image.

I knew there had to be another way because I noticed other sites are loading insecure images way faster. By trial and error I managed to create a nice setup, that first streams (not loads!) the images via an image proxy, so that they are instantly available to the end user. But to avoid proxying those images all the time, they are uploaded in the background to S3. Here’s the setup:

In this part I will explain how to setup an image proxy and load insecure images via a proxy.

In part 2 I will explain how to load those images to S3 in the background and stop proxying them forever.

Step 1: Setup a client side route for images

This step actually isn’t about setting up a client-side route in the sense that we setup Flow Router, but it’s more about defining what route the proxy is listening to / processing. We implement that later in Step 4.
Lets define the image proxy route like this:

https://www.yourdomain.com/image?user={{Meteor.userId()}}&img={{imgUrl}}

where:
u = the Meteor User id (more on that in Step 5)
img = the original image url

So an example could be:
https://www.yourdomain.com/image?u=Kfikjk89&img=http://www.flickr.com/photos/albertdros/23452079022/in/explore-2015-12-06/

Step 2: Setup Meteor to use NPM

To be able to do this we need the full power of Node, and because Meteor is basically built on top of Node, we can easily accomplish this with a little help from meteorhacks:npm. This package makes it super simple to load and use any NPM package out there. Simply follow the instructions to set it up.

Note:
I’ve encountered quite often that something gets corrupt for some reason. For me it happened when switching branch or adding/removing packages. If you run into such issue just do

rm -rf packages/npm-container
meteor remove npm-container
meteor update meteorhacks:npm

and everything gets rebuilt again.

Step 3: Add NPM Packages

Now open packages.json and add the following packages:

{
 “request”: “2.67.0”,
 “url”: “0.11.0”
}

Let me explain:
We use the Request package to make http requests and setting up our proxy. To simplify processing urls, we use the Url package.

To use these packages in your code, simply add a reference like this where you want to use them — or make them global.

request = Meteor.npmRequire ‘request’
url = Meteor.npmRequire ‘url’

Step 4: Create the Image Proxy

We create an image proxy by hooking into server requests via Meteors’ Webapp. Documentation is really poor, but it’s basically just a wrapper for Npm Connect, except that you need to call it via WebApp.connectHandlers.use.

This is the code for the image proxy:

# This proxy filters traffic on paths with ‘/image’ and takes 2 required params:
# img = an image url
# user= a Meteor user id
WebApp.connectHandlers.use((req, res, next) ->
 request = Meteor.npmRequire ‘request’
 url = Meteor.npmRequire ‘url’
# only process image urls
 return next() unless url.parse(req.url, true).pathname == ‘/image’
 queryData = url.parse(req.url, true).query
 
 return next() unless queryData.img? and queryData.user?
 userId = queryData.user
 img = queryData.img
 # check if user exists
 u = Meteor.users.findOne _id: userId
 return next() unless u?
 # pipe request to client
 x = request(img)
 req.pipe(x).pipe res
)

Our image proxy basically hooks into all the server requests, but it specifically filters for the image route (see step 1) and checks if the required params are there. I think this is quite a brutal way to accomplish this, so I was looking into server-side routing package like Picker to see if that was done any smarter. But it works in exactly the same way, by filtering url.

We also want to implement some security here, because we don’t want everyone to use our proxy right? To simplify things we include a user id in the image request url, and find this user in the database. If no user is found, then we return nothing. So that makes our proxy only available for our users (or for anyone who manages to obtain one of your user’s ids)

You could also do this via cookies, which has the advantage of not querying the database each time. I find both ways not super secure but I can’t think of anything better.

If all is fine, we use NPM Request to stream the requested image via our server directly to the client. It works like this:

x = request(queryData.img) creates a stream, which we ‘pipe’ directly into the response of the server to the client. This means, we don’t store the image on the server, it just streams through the server. It’s what makes Node I/O so powerful and you can use it right in your Meteor app. Read more about streams here

Step 5: Redirect image src to the image proxy

So now when you show images you can simply set the source to your image proxy and you’ll see your image loads like a charm, without any insecure warnings!

<img src=”https://www.yourdomain.com/img?user=Kfikjk89&img=https://www.flickr.com/photos/albertdros/23452079022/in/explore-2015-12-06/”>

But wouldn’t it be nice to automate this? Hell yeah!
It does depend on where your html is coming from. So for the sake of this tutorial, lets assume the html comes from your database.

So before inserting that html into our database, we’re going to change all the img src attributes to fetch the image via our proxy. And we make the server doing that by teaching it jQuery!

First add Npm Cheerio to your package.json file.

{
 “cheerio”: “0.19.0”,
 “request”: “2.67.0”,
 “url”: “0.11.0”
}

Now you can use jQuery on your server.

Lets assume your html is stored in a field ‘html’ in a collection Posts. Here we go:

#initialise cheerio and url npm packages
cheerio = Meteor.npmRequire(‘cheerio’)
url = Meteor.npmRequire 'url'
#get the post
post = Posts.findOne _id: postId
# load the post’s html into Cheerio
$ = Cheerio.load post.html
self = this
# find all image tags
$(‘img’).each((i, el) ->
 image = $(this)
 src = image.attr('src')
 imgProtocol = url.parse(src).protocol
 # we only want to proxy insecure images
 if imgProtocol is 'http:'
   image.attr 'src', process.env.ROOT_URL + '/i?u=' + this.userId + '&img=' + src
)
# and write the updated HTML, with proxied images, back to the database
Posts.update _id: post._id,
  html: $.html()

Now when you show your posts’ html all insecure image tags have a new src attribute and are loaded via your proxy.

Conclusion

Before Meteor I didn’t have any experience with Node. But now that I used it a bit I feel the Meteor packages are quite limited compared to the huge amount of Npm packages out there. And the good thing is, you can use those packages very easily.

It’s really cool to setup something like this —although it took me a good few days digging and searching for a solution that works. Displaying insecure images on a secure site is a common challenge, and maybe there are easier solutions out there. But I hope my solution is of use to someone.

In part 2 of this tutorial I will explain how to setup a background job that uploads those proxied images to S3. Because we don’t want to proxy them forever now, do we?

Stay tuned!