Rescuing myself from the email monster with JavaScript Rescuing myself from the email monster with JavaScript Jun. 23rd, 2014 Taylor Smith

Rescuing myself from the email monster with JavaScript

June 23rd, 2014

We here at Four Kitchens do love us some email. Last week, I self-awarded a prize for having achieved 4,000 unread items in my inbox, with another 3,000 read items sitting in my inbox for no reason. I could keep going for the record, but I thought I’d attempt to use nerdiness to take better control.

(Ultimately, it’s a chore that simply has to be done, but maybe it can be easier.)

TL;DR: If you just want the source code, grab it on GitHub.

Filters feel limited

I came to Gmail from Exchange; Filters have always seemed less powerful than Rules.

My gripes with Gmail Filters:

  • Search is limited, seemingly, to word and number characters; special characters have no effect in searches, so I can’t isolate “[ category ]” strings we use in subject lines.
  • Complex groups of boolean logic have a tendency to produce imprecise results and there’s no way to have “additional criteria” (even if multiple fields are used in the advanced search/filter builder, they’re flatted into a single query)
  • Other header information or metadata can’t be used in the search criteria (like mailed-by to help weed out notifications “from” a user sent on their behalf by another service)
  • Filters cannot be run on sent mail (for the purposes of auto-labeling)
  • Filters cannot be run on a delay
  • No regular expression matching

Ta-Daa! Gmail can be scripted with JavaScript!

Checkout Google Apps Script and create a new blank project.

Google Apps Script: New Project

This is just JavaScript, but it runs server-side within Google Apps and can be run on regular intervals or on specific triggers. You do not have to be logged in with a window open to make this work!

Step 1: Migrate filters to JavaScript for more power

Goal: Label everything, both incoming and outgoing. Additionally, some theads are starred, marked as /(un)?(important|read)/, or immediately auto-archived.

A new function is created for tagging messages. I compile a list of useful variables and then move straight to categorizing:

Timely Messages generally require direct action, quickly. They are added to the label ‘~/Announcements’ by thread.addLabel() and also starred using message.star().

Gotchas:

  • The addLabel() function requires a Label object, not a string. Such an object can be obtaind using GmailApp.getUserLabelByName().
    • Be sure to include ‘parent/child’ if your labels are hierarchical. (In this case, ‘Announcements’ is a child of ‘~’).
  • When using string.match(), be sure to add the i flag at the end of the pattern to ignore case, since authors may be inconsistent with case.
  • If you see an error like Cannot retrieve (line X, file "Code") where X is a line containing a getUserLabelByName call, the most likely cause is that Gmail couldn’t find that label.

Whereabouts emails are generally uninteresting since I work remotely most of the time. They’re all flagged as ‘~/Whereabouts’ and, if they don’t appear to indicate that the sender will be unavailable, they are archived immediately:

Regex on line 2 visualized:

Whereabouts Regular Expressions

Google Calendar emails can be identified by what the subject line starts with:

Regex visualized:

Google Calendar Regular Expressions

Client emails get sorted as well. We’re a little lax in the formatting of those tags, but regex makes that easier:

Regex visualized:

FPL Regular Expressions

This matches [fpl], [full plate], [full plate living], [fullplateliving], and various others, as well as any email sent to/from @fullplateliving.org.

In general, the function contains three pieces:

  • If statements testing timeliness or general discussion topics
  • If/else statements testing for one of any application notification (Google Calendar, GitHub, JIRA, Notable, etc.) since a thread won’t be from multiple
  • If/else statements testing for one of any client name, since a thread is unlikely to pertain to multiple clients directly, although I may change this.

This allows a thread to end up with multiple labels at the expense of running a little slower, but the load is reduced by being conservative with the triggers (Step 3).

Step 2: Script email expirations

My second function will archive threads that have dated out. Since the autoTagMessages() function has nearly everything categorized, I’ll base retention and expiration off of labels, thread ages, and whether or not the thread is read. This can be done by executing Gmail searches programmatically using GmailApp.search().

Set up the searches as standard search queries:

Then run the searches and, in batches of 100 (batchSize), archive the resulting threads:

Gotchas: That AND (-is:starred) at the end of the search string doesn’t always work. Sometimes threads with starred messages are archived anyway. But there is a way to fix that:

(I didn’t say it was a graceful way… Investigating better options…)

Step 3: Setup triggers (like Cron for your inbox)

Now I have two functions:

  1. autoTagMessage() – given a thread, label it appropriately.
  2. autoArchive() – search for email that can be archived and do so.

Trigger autoTagMessage hourly on new email only

Google Apps Scripts can be triggered routinely, but unlike Outlook, there is no ‘run as a message is received’ option. This can be emulated by:

  • having Gmail filters assign one label to every incoming email, and then
  • processing all messages in that label on a regular basis (5 minutes).

I assign the “Prefilter” label to all incoming messages by matching against having an @ in the to field. All other filters were exported and deleted. Using the Label settings, “Prefilter” can be hidden from your inbox view so you don’t see it.

Prefilter

Unexpected benefit: I noticed that this “Prefilter” label is applied to all outbound email as well (perhaps because it matches only against the to field), allowing messages I send to be auto-labeled with no additional work!

Then, in a new function, I get those threads and tag them:

Next, amend autoTagMessages() to remove that label, and, if a thread has multiple messages, abort. This will prevent re-labeling an entire thread for any new messages in it (which would only be annoying in the case that a message is starred; for example, replies to a [timely] thread would be starred otherwise).

Now I have two functions that can be run on a regular basis, so let’s do so. Under the “Resources” menu, click “Current project’s triggers” and add these:

Triggers

  • autoArchive() can run hourly (or less frequently, honestly).
  • batchIncoming() must run very frequently. I chose 5 minutes instead of 1 so that it wouldn’t start again before the last execution has finished.
    • Google Apps Scripts will timeout and abort at five minutes, although I haven’t hit that limitation.

Declare a reset, then profit

Be sure to warn folks when you’re about to purge a few thousand threads from your inbox. Then sit back, keep up with what you can using the auto-labeling help you’ve built, and let Google Apps Scripts help you.

Next steps

I’m still working to:

  1. Find an efficient way to filter threads with starred messages out of a GmailApp.search() result, so that I don’t have to do that stupid “un-archive any starred threads” maneuver in autoArchive(). There is a method thread.hasStarredMessages(), but using that would require iterating over each thread in the result-set, which seems expensive for an otherwise batched process.

Additional reading

Comments