welcome at SebWalak.com

AppScript - How to automatically groom Gmail account?

categories: /software-development
tags: #js   #cloud   #google-cloud

This post shows my example implementation of auto-delete solution that will traverse my Gmail account in search of qualifying messages and remove them.

If you have ever owned some of the cheap CCTV cameras you might have chosen to automatically send motion snapshots and alerts to your email address. It can work but can overload your email account very quickly. Some people will say that I haven’t used the right tool for the job and I completely agree with them. However, if you are looking to deploy a solution quickly and cheaply, sometimes that’s the only option available.

So, the reason it was a problem for me (apart from poor choice of technology) is that I insist on using a PC-based email client rather than Gmail’s web interface. Once the number of emails (each with some photos) goes beyond 30’000 the email client just grinds to a halt. However, I like the fact that, while I make myself coffee, my email client fetches the emails so that I can quickly look through the snapshots. When using web interface the buffering makes experience more jerky.

I have been looking into ways to construct some automatically triggered rules around this particular Gmail account, in order to groom it with minimal input from me. I have failed to find anything built-in into web interface or my email client. I’ve decided to learn some AppScript and run this on a timer from within Google’s infrastructure.

Kinda cool to use one Google service to sort out another :)

In order to let you to play with the concept of running AppScripts, have a look at the following function and then follow the mini tutorial steps below. If you are already setup jump to the end to have a look at the finished function.

function playWithGmailQuery() {
  var userId = 'me'
  var threadList = Gmail.Users.Messages.list(userId, {
    q: '\
from:(camera1@domainUsedByYourCCTVSetup.com) \
subject:(snapshot) \
after:2014/1/1 \
before:2017/8/14',
    maxResults: 50
  });
  if (threadList.messages) {
    Logger.log("Found following messages");
    threadList.messages.forEach(function(message) {
      var fullMessage = Gmail.Users.Messages.get(userId, message.id);
      Logger.log(fullMessage.snippet);
    });
  }
}

Step 1: Make sure the browser you are going to use is currently logged-in to the Google account which is associated with the email account you want to automatically groom. The reason is that the above script is operating on that email account. It is controlled by userId parameter throughout the Gmail APIs, here it is set to 'me' which will mean your account, when you run this routine.

Step 2: Go to https://script.google.com/. You will see a window which looks something like this.

AppScript first look

Step 3: Name your project by clicking “Untitled project”. I’ve called mine “gmail-grooming”

Step 4: Copy&Paste the above snippet of code over the default example with “myFunction”. You should end up with similar view to this

Simple example of AppScript code logging snippets of Gmail messages

Step 5: Red asterisk next to tab’s name indicates unsaved code. Save it with a click on a floppy disk icon (I’ll shed a tear in the meantime as it gets me all sentimental).

Step 6: If this is your first encounter with Google App Script then you will have to complete an extra step, turn on Gmail API. To do this click menu item “Resources”, then “Advanced Google Services …”. On the dialog that pops up, turn on Gmail API

Series of switches, one per Google API, Gmail API switch is turned on

As per the message on the above dialog you also have to enable that in the console. Click the link at the bottom.

Now select Gmail API on the following screen

Enable Google APIs in Console

and then option “Enable” will show up, which you choose.

Then you will have to grant third party app (well, yours in this case) permissions to operate your chosen Gmail account. It will need grant to read, send, delete and manage your email.

Step 7: Now the Gmail API is turned on and permissions to your app granted we can start the running the app. First select a function that you want to run. We only have one named playWithGmailQuery so select that name from the drop down right above the code. Then click the play icon and enjoy.

Nothing seemed to happen? It is because all this application does is log the snippets of emails that match our query defined in line 4. Also, don’t forget to alter the query to match some of your emails.

There could be another reason as well. Your query could have been so generic that it just takes long to execute. Pay attention to the top area above the code, you may see the yellow notification saying that your application is still running

Notification about running function

To see the log press Ctrl + Enter (you can also use menu to open log). There is fair chance your log will show nothing as the query won’t match any of your emails. To demonstrate how populated log window may look, I’ve modified my query to match some of my emails

'from:(JetBrains.com) subject:(Newsletter) after:2014/1/1 before:2017/8/14'

which produced the following log:

Example listing of AppScript logger output showing snippets of emails

Step 8: It’s all great but the routine was supposed to be triggered automatically, wasn’t it? You are right. Above the editor panel you’ll find an icon resembling a timer of some sorts. It enables setup of triggers

Dialog allowing to choose an AppScript function along with its trigger type

For the above options the Google infrastructure will run your routine playWithGmailQuery every hour.

And the following is the code that you can use for removing Gmail messages according to your own rules:

function gmailGrooming() {
  const userId = 'me';  
  const detailedMessageLoggingEnabled = false;
  
  const queries = [
    constructQueryString('JetBrains.com', 'Newsletter 2017', '2014/1/31', 50),
    constructQueryString('cctv-1@example.com', 'snapshot', '2016/6/15', 30)];
    
  var totalCount = 0;
  for (var i in queries) {
    var count = forEachMessageMatchingQuery(
      userId, 
      queries[i], 
      function(message) {
        if (detailedMessageLoggingEnabled) {
          const fullMessage = retrieveFullMessage(userId, message.id);
          logInfoAboutMessage(fullMessage);
        }
        removeMessage(userId, message.id);
      });
    
    Logger.log(count + ' messages matched query "%s"', queries[i]);
    totalCount += count;
  }
  Logger.log(totalCount + " messages matched in total");
}

/*
fromSender - email to be deleted has to be sent from 
    an email address that contain this string. 
emailSubject - email to be deleted has to contain 
    this string in the subject (as well as fulfil the above criterion). 
ignoreBeforeDate - read as "do not touch message if is older than this date", 
    even if email matches the 1. and 2. it will not be deleted 
    if it was received before this date, 
preserveLastDays - even if email matches the 1. and 2. 
    it will not be deleted until its at least this many days old
*/
function constructQueryString(fromSender, emailSubject, ignoreBeforeDate, preserveLastDays) {
    var date = new Date(new Date().valueOf() - (preserveLastDays*24*60*60*1000));
    var dateString = date.getFullYear() + '/' + (date.getMonth() + 1) + '/' + date.getDate();
    return 'from:(' + fromSender +
      ') subject:(' + emailSubject +
        ') after:' + ignoreBeforeDate + 
          ' before:' + dateString;
}

function logInfoAboutMessage(fullMessage) {
  Logger.log('Removing message with id %s, dated %s, snippet "%s"', 
             fullMessage.id, 
             extractDate(fullMessage), 
             fullMessage.snippet)
}

function extractDate(fullMessage) {
  return fullMessage.payload.headers.filter(function(header) {
    return header.name == 'Date';
  }).map(function(header) {
    return header.value;
  });
}

function forEachMessageMatchingQuery(userId, query, callback) {
  var threadList = Gmail.Users.Messages.list(userId, {
    q: query,
    maxResults: 50
  });
  if (threadList.messages) {
    threadList.messages.forEach(callback);
    return threadList.messages.length
  }
  return 0
}

function retrieveFullMessage(userId, messageId) {
  return Gmail.Users.Messages.get(userId, messageId);
}

function removeMessage(userId, messageId) {
//  Gmail.Users.Messages.remove(userId, messageId);
}

Note: The function removeMessage contains a single commented out line which prevents the call which actually removes the message. Uncomment once you know the query does what you want. Check the log for total message count matching your queries and enable detailed message logging to review what messages match.

If you want to extend the functionality by querying different parameters here is something you might like.

Thumbnail