Using Twilio to filter pictures

Photo filters are a big thing today. Everyone seems to love Instagram. So this post is going to show you how to get filters working via MMS with Twilio and a Node.js backend. Disclaimer - This definitely works with jpegs, but I’m not sure about pngs, as everything I send to twilio ends up as a jpeg.

This assumes you already have a Twilio account, which you can get at Twilio. The next thing you'll need after that is a couple of node modules installed via npm.

npm install hapi twilio node-uuid jpeg-js png-js file-type image-size inert  

Hapi is the framework we're using to handle our requests.
node-uuid is just to create filenames that we are sure are unique.
jpeg-js lets us encode and decode jpegs files and png-js does the same for png.
file-type and image-size just allow us to apply filters to our images.
inert is the file handler for Hapi.

Make sure your Twilio credentials are stored in environment variables.

const twilio = require('twilio');  
const client = new twilio.RestClient();  

Next we create a few helper methods.

This method just downloads the file for us and stores it on the file system.

let download = function(uri, filename, callback){  
  request.head(uri, function(err, res, body){
    request(uri).pipe(fs.createWriteStream(filename)).on('close', callback);
  });
};

This method decodes our image into an array of pixels. It will handle jpeg or png.

  let fileData = fs.readFileSync(fileName);
  // Get file type
  let metaData = fileType(fileData);
  console.log(metaData);
  let imageData;

  if (metaData.ext == 'jpg') {
    imageData = jpeg.decode(fileData);
    callback(imageData);
  } else if (metaData.ext == 'png') {
    png.decode(fileName, function(pixels) {
      var size = sizeOf(fileName);
      // Convert to jpeg.  Writing back out to png was a pain.
      imageData = {
        data: pixels,
        width: size.width,
        height: size.height
      };
      callback(imageData);
    });
  }

Once we have an array of pixels, we just pass those pixels to our filter; in this case grayScale.

  imageData.data = filter.grayScale(imageData.data);

This is the Hapi route to handle the incoming message:

server.route({ method: 'POST', path: '/receiveMessage', handler: function(request, reply)  

This is the function that runs upon receiving a message:

     // Download media
    download(message.MediaUrl0, fileName, function() {

      // Apply the filter to the media
      applyFilter(filterName, fileName, function(filteredFileName) {
        // Create the response and return the filtered media
        const twilioMessage = {
          body: "Filter applied!",
          to: message.From, 
          from: message.To,
          mediaUrl: 'http://curs.es:8088/'+filteredFileName
        }
        createMessage(twilioMessage);
      });

This of course isn't doing any sort of error handling which would need to be added. This is the function to actually send the message.

function createMessage(twilioMessage) {  
  client.sendMessage(twilioMessage, function(err, twilio) {
    console.log(twilio.sid);
  });
}

This is all we need to do to get the processing setup for the messages. Now we want to add some filters. I got a lot of this data from http://www.html5rocks.com/en/tutorials/canvas/imagefilters/ so take a look if you want to understand how these filters work.

I moved my filters to a different file just to make things a little bit cleaner.

This is the grayscale filter.

    // Go pixel by pixel and change to grayscale
    for (let i = 0; i < imageData.data.length; i += 4) {
      let r = imageData.data[i];
      let g = imageData.data[i+1];
      let b = imageData.data[i+2];

      let v = 0.2126 * r + 0.7152 * g + 0.0722 * b;
      imageData.data[i] = imageData.data[i+1] = imageData.data[i+2] = v;
    }

This is the threshold filter.

  // Go pixel to pixel and change to pure black and white
    for (let i = 0; i < imageData.data.length; i += 4) {
      let r = imageData.data[i];
      let g = imageData.data[i+1];
      let b = imageData.data[i+2];

      let v = (0.2126 * r + 0.7152 * g + 0.0722 * b >= threshold) ? 255 : 0;
      imageData.data[i] = imageData.data[i+1] = imageData.data[i+2] = v;

    }   

The last thing we need is a route to serve up our altered images to Twilio to send back.

server.route({  
  method: 'GET',
  path: '/images/{file}.jpg',
  handler: function(request, reply) {
    reply.file('images/'+request.params.file+'.jpg');
  }
});

That's it! Feel free to try it out by sending an MMS message to 702-666-0736.

Dig in and take a deeper look at https://github.com/slooker/twilio-image-filter.