Using Meteor with Postgresql 😮

8 min readApr 16, 2021

I’ve been using Meteor for many years now and no matter what other framework I try, I keep coming back to it. The reason is simple, unmatched development productivity. Somehow Meteor got a bad name for not being scalable, being slow, and only being suitable for MVP development. Fortunately that was never the case.

Meteor was never slow, wouldn’t scale or was only suitable for MVP. It’s just a framework with a personality

Perhaps it was caused due to a lack of understanding the limitations of the framework. And probably it was caused by an exodus of community members to other “hot” technologies such as Next and React in 2016.

And yes, Meteor is definitely to blame for it as well, as they raised millions to focus on Apollo (and what a success it is!) and in the process pretty much neglected Meteor.

But, since Meteor was acquired by Tiny, the development of the framework is very much alive, fast paced and really moving in the right direction. Not to mention the many companies growing successfully while using Meteor: Qualia recently became the first unicorn using Meteor, ConvertCalculator is growing rapidly and PostSpeaker growing strong, to name a few.

So what makes Meteor 😍 (imho)

super fast app setup and modern build process (without any config)
awesome support for React, Vue and Svelte
extreme fast API development with Meteor methods. Simply write function on the server and call them from the client. It really is the killer feature of Meteor.
Meteor Cloud for hosting, a thin auto-scaling layer on top of AWS, eliminating devops
great testing support
and Meteor just being a thin layer on top of Node, so the framework is never in your way. Use anything Node if required.

Why Meteor needs Mongo (and why you shouldn’t care 🤷🏻‍♂️)

There is one thing that still kind of limits Meteor, and that is its tight integration with Mongo. It’s not that Mongo is bad, it’s just that most data is relational and Mongo hosting is expensive due to Mongo almost having a monopoly these days. Meteor’s MiniMongo (db queries in the browser) is cool but it has (performance) limitations making it hardly useful. And honestly, you can do without.

The actual reason Meteor has a tight integration with Mongo is because of its real-time capabilities. How it works is that the client subscribes to a “publication” on the server, and the server uses Mongo’s oplog for observing collection changes. If anything changes, those changes are then pushed automagically to the client (it uses Meteor’s DDP protocol, a layer upon websockets)

This is a technology that was very cool when Meteor launched, and it has its uses. But it’s also the main bottleneck for Meteor’s performance. It adds load to the server, and to the database. And honestly, real-time can be solved in many ways these days.

As a rule of thumb: don’t use Meteor’s publications, they will always cause you headaches. Simply use websockets (or Meteor’s DDP) to inform the client of changes, and then have the client fetch data by API.

Don’t use Meteor’s publications, it’s useless by default due to performance implications and will always cause you headaches.

In other words, Meteor must stay tightly coupled to Mongo, because of its real-time publications offering. But that technology is quite useless by default due to its performance implications, and therefor should not be used.

Postgres! 🤩

Now that we’ve concluded that publications are a no-go, we get the amazing opportunity to drop Mongo and use Postgres!

This guide gives you a general overview of how to do it:

Create 2 databases
Setup db schema using Sequelize or anything else you like)
Remove Meteor packages that depend on Mongo
Roll our own auth (as Meteor’s user account system can’t do without Mongo)
Fix running tests
Bonus: publish-subscribe

Let’s get started by creating a Meteor React app! 🚀

meteor create --react meteor-postgres

1. Create 2 databases

You will need to create 2 databases (locally), one for your app and one for testing your app — unless you don’t care about overwriting your database constantly when running tests.

2. Setup db schema

You now have the freedom to use any ORM for Postgres, or use Knex or do native SQL queries. Anything you’d do normally with Node. For this guide we use Sequelize to run migrations.

3. Remove Meteor packages that depend on Mongo

Now it gets interesting, you will have to remove quite some packages that depend on Mongo:

// remove all mongo deps
meteor remove mongo autopublish mobile-experience reactive-var insecure react-meteor-data// enable hot reload again
meteor add reload autoupdate// enable testing
meteor add meteortesting:browser-tests meteortesting:mocha meteortesting:mocha-core

Now get rid of any Mongo dependencies in the scaffold project:

Replace server/main.js with:

import { Meteor } from 'meteor/meteor';Meteor.startup(() => {
   console.log('server started')
});

Remove imports/api/links.js and replace imports/ui/Info.jsx with:

import React from 'react';export const Info = () => {
  return (
    <div>
      <h2>Learn Meteor!</h2>
    </div>
  );
};

Also note that Meteor’s account system is now removed. You will have to roll your own authentication.

Run it to test it all works well without any errors meteor run --port 3050

4. Roll your own Authentication 💪

Here it gets interesting. As we removed the Meteor account system, we need set it up again in a way that’s similar to Meteor’s native account system. The reason we’d want to use a similar implementation, is that we want to keep using all the nice goodies of Meteor, such as Methods. For example, we want to be able to keep doing this on the server:

Meteor.methods({
   getCurrentUser: function() {
      if (!this.userId) throw new Meteor.Error('unauthorized')      return Meteor.userId()
   }
})

The Meteor authentication system is hardly documented, but by digging in the code I figured out the following schematic of how it works.

You must setup all the different parts on the client and the server to make it work and I’ll walk you through it in the next sections.

The best thing is, now Meteor accounts system isn’t a black box anymore, you will have full control of how you roll your authentication.

👉 Server

First setup a login method to check user credentials submitted by the client and generate a sessionToken if those credentials are correct. In other words, do what you would do anyway if you would write your app in Node or any other framework. And don’t store plain passwords in your database!

After successfully generating the sessionToken and storing it in the database, you must set the userId on the connection, like this:

Meteor.methods({
   login: function(email, password) {
      // check credentials, get User from database
      // generate a sessionToken and store in database      // now set the userId on the server-client connection
      this.setUserId(User.id)      // return the generated sessionToken to the client
      return sessionToken   }
})

Note that setUserId() only accepts a string, you cannot set the full user object (unless you Stringify it, which would allow you to put additional information on the connection).

Now if you perform Meteor.call from the client, each Method function invocation will have access to this.userId, which now always resolves to the current logged in userId 🥳 Use it as you would always.

💻 Client

Good, the client is logged-in. Unfortunately this only stays that way until you refresh the client, or if you restart the server. Which is definitely not what you want.

The first thing you need to do is to store the sessionToken returned from the login call in localstorage.

Now you will add the following code to the client:

import { DDP } from 'meteor/ddp-client'DDP.onReconnect(() => {
   const resumeToken = localStorage.getItem('sessionToken')
   
   if (sessionToken) {
      // woot woot! client is already authenticated
      // -> resume the session      Meteor.call('resumeSession', sessionToken)
   } else {
      // login first
   }})

We make use of the fact that Meteor establishes a DDP connection as soon as a client connects with the server. When that happens, the onReconnect callback is triggered.

In that callback we have the client check for a stored sessionToken in localstorage. If it has it, it means the user is already authenticated. Simply resume the session by sending the token to the server. To do so, we need to make a resumeSession method on the server

Note that this is the exact same mechanism that Meteor uses itself for its account system. We simply replicate it.

👉 Resume sessions

Now add the following method to the server

resumeLogin(sessionToken) {   // get sessionToken from database
   // get User from database based on sessionToken   if (!User) throw new Meteor.Error('unauthorized')   // if there's a user, set its id again on the connection    this.setUserId(User.id)
   
   // optionally generate a new sessionToken
   return sessionToken}

Whenever you restart the server or the client, the session will be resumed as soon as the client-server connection is restored, and the user will be logged in. And that’s how you roll your own Meteor authentication 🚀

Well almost, add this logout method as well

Meteor.methods({
   logout: function() {      // clear the userId on the connection and 
      // make sure the remove/invalidate the sessionToken in the 
      // database
      this.setUserId(null)
   }
})

And make sure to clear the sessionToken from localStorage on the client after logging out!

If you wish to enable Meteor.userId() on the client (not really necessary but if that’s what you’re used to…) enable it on the client with this line of code:

Meteor.connection.setUserId(userId)

Now you’re really done, and you can make the actual authentication and session management as simple or sophisticated as you like 🥳

👉 Fix running tests

When Meteor runs tests with Mongo, it starts up a separate database to make sure your development database isn’t wiped. To replicate this behavior, you’ve already created a 2nd Postgres database for testing. Now all you need to do is enable a couple of npm scripts in package.json, (this example is using sequelize but you get the gist of it)

// migrates database up
"migrate:up": "npx sequelize-cli db:migrate",// undo all migrations, basically dropping all tables
"migrate:drop": "npx sequelize-cli db:migrate:undo:all",// run test
"test": "source .env-test && npm run migrate:test:drop && npm run migrate:test:up && TEST_WATCH=1 meteor test --full-app --driver-package meteortesting:mocha -p 3062",

The last line needs some explanation. First it sources the database credentials from .env-test, making sure the app connects to the test database and not your main database.

Then it drops all migrations and immediately applies them again. This makes sure that your test database is reset, but that it also has the latest schemas applied.

Next, a new Meteor instance is started on another port, watching your code and rerunning tests upon code changes.

Now you can develop and run your tests at the same time 🚀

👉 Bonus: Publish-Subscribe

If you need publications, to push data to the client when changes happen in the database, you can use Postgres’ native Notify functionality. Not diving into it now, but here’s an example to get you started.

Another option is to use Sequelize’s hooks to push messages over websockets channels, or roll your own notification system. It’s not that difficult really.

💥 Done! 🕺

Now Meteor runs smoothly without Mongo!

You get all the power of Postgres, still keeping everything that’s awesome about Meteor.

Using this stack, you’ll find your development productivity to be incredible, backed by a database with superpowers. You will spit out APIs faster than Jay-Z can rap, and you can still have real-time capabilities for your app.

Off to the 🚀🌗!