The problem is this: with Outlook we will be limited to 1000 messages in our in-box; sadly, many of us have tens of thousands of emails in our old GroupWise mail. Even after a fairly rigorous slash and burn mission, hacking out all of the low hanging fruit, there will be many thousands remaining and I don’t want to lose that information. It might be useful to search and find how I set up a Zebra bar code printer in 2003, no?

A bundle of different freeware glue tools came to my rescue. Read on to hear about the toolset that has made it so I can keep those messages for years to come.

Possible Solutions

Right out of the gate, I began looking for ways to migrate messages from one mail client to the other. Some apps have this built right in, and if not, there are scripts and utilities out there to do this; but I was hampered by a few key facts:

• I have no control over the email clients and their configuration. Even if there is a menu option for exporting GroupWise messages from version 7.2, I’m stuck at 6.4 and cannot use that option.
• GroupWise is a minor player in the email world. I’m not sure if Outlook would import from GroupWise, but I doubt it.
• They are replacing the client in one shot. There will be no interim period where both GroupWise and Outlook will be available.
• There is no getting around the hard limit of 1000 messages.
• I don’t want to spend money on this.

With these constraints in mind, I immediately thought about PDF documents. I then considered the following questions:

• How do I convert my email to PDF?
• How can I do this automatically with thousands of emails?
• Once I’m done, how do I search these documents?

Here’s what I did:

Conversion to PDF

The first part was easy. I downloaded one of the many free print-to-PDF products available.

I chose PDFCreator, because I am familiar with its use and I know that it does not munge the fonts.

Like many other PDF generation utilities, PDFCreator functions by providing a virtual printer to which any application can print. For example, to make a PDF of a web page, you use the Firefox Print menu and select PDFCreator from the drop-down list of available printers.

You are provided with a list of metadata fields that you can fill in, and these fields are used in the PDF generation.

Here’s what the PDFCreator screen looks like:

A word of caution: PDF Creator is free, but you must be careful to deselect their spammy toolbar options in two different places during the installation process. I don’t like software that comes with preselected toolbars to install (even nice ones like Google’s) because I’m certain that 95% of the folks who actually install the toolbar would never have chosen to do so if it were unchecked by default.

Running Everything Automatically

This was the interesting bit. I work with Windows machines at work, so there was no AppleScript option available. So I did the next best thing: I used AutoIT.

I will warn you that AutoIT is pretty much the Windows analog of AppleScript, without the cutesy pseudo English syntax. In other words, you will need to roll up your sleeves and get your hands a little dirty in order to put together a decent AutoIT script.

The payoff comes when you finish your work and compile it into a tight executable that you can share with your friends, allowing them to automate some complex series of button clicks and copy/paste operations.

I walked through the manual process of exporting an email to PDF and listed each action:

• Get the date, sender, and subject
• Create a filename based on date + sender + subject
• Launch the Print dialog
• Select PDFCreator
• Fill in the Document Title, Creation Date, and Subject in the PDFCreator dialog
• Fill in the full file path in the Save dialog

In addition, I wanted to make the script a little better by adding the following:

• Check that user has PDFCreator installed
• Verify that GroupWise is running and that the user has selected one or more messages
• Prompt the user for a target directory before processing the messages
• Sanitize the filenames by replacing illegal characters with underscores and truncating to meet maximum filename and path length in Windows
• Skip over files that have already been generated, quickly, so that one doesn’t need to worry about accidentally selecting messages that were already printed

There were other adjustments needed, but the process was the same: run the script, hit a problem, tweak the script a little to address the problem, and repeat.

Here’s a little bit of the AutoIT script:

You can see that it is a bit more intense than AppleScript, but remember that the full script wasn’t written in one go. I had a little short ten-line script that I kept tweaking as small problems cropped up until I had adjusted things to my liking.

Note that this is a GUI macro language. The machine starts clicking and typing away right in front of you and you probably shouldn’t interfere until your script finishes.

As of this afternoon, I have generated around 4,000 PDF documents for my email messages.

Searching All of Those Documents

This was the easiest part. These days there is an excellent tool available for searching documents on your desktop: Google Desktop. This product indexes every useful file on your desktop and provides a full Google search with a quick double-tap of the <control> key.

So you can enter a search like “Zebra bar code”

And the results look exactly like a Google web search, but it’s showing your desktop files. And you can see inline previews too.

Macintosh users can install Google Desktop as well, but all of these files should already be indexed and searchable by Spotlight.

Closing Thoughts

Whenever I reach for tools like this I feel a twinge of guilt—it’s outright hackery, isn’t it?

But there is a place for quick and dirty jobs in every workplace. I needed to get my files from one place to another, one time only. It just didn’t make sense to spend money or time on a more elegant solution.

Play around with each of these tools a little. Especially AutoIT—it’s a handy Swiss Army Knife to have at your disposal.

Of course, it is only a matter of days, or even hours, before the new glow has worn off and I begin to grouse about the things I wish they had done and I grumble about the nit-picky little bugs that are always there.

Even when the program works perfectly, we often want to use it in a work flow, using each program for what it does best.

Work flow is inherent in the kind of work that we do when scanning, indexing, searching, filing, tagging, and backing up all of our documents, photos, music, and video. Once you are committed to digital media, you will find that you often need to cobble together different programs in order to do away with some of the tedious manual labor.

Scripting tools can be as simple as macro recorders, as fancy as drag-and-drop pipelining tools, and as complex as full programming languages. Choose one that fits you and use it to bridge the gaps in your work flow.

An Example

I have never had a harsh thought about my sheet-fed scanner, the venerable ScanSnap S510m. But one thing has always annoyed me about the software package they provide: you can’t start scanning a new document until the OCR process is complete on the first.

What happens is this: you scan in a document; ScanSnap Manager receives it as a PDF and immediately passes it to Abbyy FineReader for OCR. All is good at this point.

But, the OCR process can take anywhere from 10 seconds to a few minutes, and if you were to scan in another document, FineReader complains:

This is where a little bit of AppleScript magic smoothed over this minor work flow annoyance.

I found that you can attach a bit of AppleScript as a “folder action” to a folder, and when a file appears in that folder, your AppleScript gets called. All I needed to do was figure out a way to get AppleScript to do the following:

• Wake up when a file appears
• Grab the first unprocessed PDF file
• Tell FineReader to start up, if it isn’t running
• Tell FineReader to open the file
• Wait for the processing to finish
• Go back to step 2 until all files are done.

It wasn’t exactly trivial—FineReader doesn’t actually have any special AppleScript support built in. But after a few experiments (i.e. a full weekend of fussing around) it was working smoothly.

Now I can scan as many documents as I want, knowing that OCR is happening at its own pace in a totally different flow.

Some day, when I have cleaned up the embarrassing bits of the code, I will share it here for fellow ScanSnap users.

What scripting tools are out there

The automation tools you use will depend on what software you are working with and what operating system you are using. On the Macintosh side, AppleScript and Automator will probably cover the bases. Things aren’t so pretty on the Windows side, but if you are willing to get a little dirty, Windows scripting can work miracles.

Whatever tool you use, I recommend finding as many simple examples on the Web as you can and running them and tweaking them slightly until you make them do what you want.

I am not an AppleScript programmer; I simply downloaded other people’s sample scripts that did file manipulations and poked and prodded them into shape.

AppleScript

In the Macintosh world, the old standby has always been AppleScript, with its quirky syntax that attempts to look like English. It is surprisingly easy to do some fairly heavy lifting with a minimal of effort.

In fact, though I find its syntax challenging and condescending at the same time, I heartily recommend AppleScript. The key is that Apple has built in AppleScript support to most of the main Macintosh applications.

For a taste of what AppleScript can do, here’s a whole mess of cool scripts for automating iTunes.

Imagine running a script that builds playlists for you or tags files automatically or finds duplicates and marks them.

This is the main AppleScript site, and here’s a tutorial from Apple.

Automator

Another Macintosh product, that provides a more graphical approach to hooking your apps together, is Automator. Here is an article that gives a good overview of the tool.

This is what a simple work flow looks like in designer mode:

You can see more or less what is going on here: You use a predefined set of actions, on the left side, to build up a pipeline on the right side.

In this case, the new automation action will get the selected documents, sort them, combine them into one PDF document, and then open that document in a PDF viewer.

I have always been a little confused with the utility of this product: it’s great if your application provides Automator actions, but what do you do if your app is like FineReader, with no special hooks built in?

Nevertheless, with Automator, you can do some pretty fancy things with many common tasks such as file management, email, and browsing. And there are plenty of custom Automator actions out there for the downloading.

Both Automator and AppleScript are Macintosh OS X built-ins.

For the serious Mac user, there is Bash Unix shell scripting, but that’s all I’m going to say about that. It’s there if you want it!

Windows Scripting

Things are somewhat different on the Windows side.

On the positive side, I believe that Microsoft’s COM object architecture makes it possible to script some pretty fancy things using off-the-shelf software.

In addition, the Office suite has amazing macro facilities that have been used by many as a full programming language, as the core of many successful commercial products.

But Windows just doesn’t seem to have anything analogous to AppleScript for gluing things together without getting your hands too dirty.

As a trivial example of the power of the Windows COM architecture, this short script will launch Excel, create a fresh spreadsheet, and automatically feed in some data. To try it out, save the script to your desktop and double click it.

This is what I saw on my machine when I ran the script:

Just like AppleScript on the Macintosh side, many Windows applications provide hooks for scripting.

The down side is that the scripting language is much more like a programming language, so it’s more comparable to writing shell scripts than to using a drag-and-drop automation tool like Automator.

But when you do the same thing over and over every day, it may be worth it to roll up your sleeves and get down and dirty with some VBScript.

An added benefit of this approach is that the language shares much of its functionality with Visual Basic for Applications, found in Word and Excel macros. If you are writing standalone Windows scripts, you can also write very complex Word and Excel macros. You also have the ability to do much of the same application automation within Word/Excel macros.

AutoIT

This is a freeware tool that many use for scripting their applications.

From the product website:

• Easy to learn BASIC-like syntax
• Simulate keystrokes and mouse movements
• Manipulate windows and processes
• Interact with all standard windows controls
• Scripts can be compiled into standalone executables
• Create Graphical User Interfaces (GUIs)

What more could you ask for?

Closing Thoughts

There are other tools out there; don’t be afraid to check them out and try some of the examples. If you can automate just one annoying task that you do every day, it may be worth the hassle of spending a weekend writing the script.

Epilogue

Yesterday at work I finally rolled up my sleeves at work to tackle a nagging document problem that had been lingering for months.

We have a large quantity of PowerPoint slide decks that have critical company information embedded within. I work in the pharmaceutical industry, so many of those PowerPoint slides had chemical structures for various drug molecules embedded within as OLE objects.

The problem was that we wanted to extract that knowledge (the molecule files) electronically and text mine the slides, but our tools did not support PowerPoint. They did, however, support Microsoft Word.

How could I convert the slides to Word documents? There are dozens of ways on the Net to go in the other direction, but the only way to do what I wanted resulted in the OLE objects being converted to pictures—no good for my purpose. I did find that a copy/paste of each molecule from PowerPoint to Word worked correctly, but who wants to do that by hand with hundreds of documents?

After weeks of trying to find a real Microsoft developer at work to do the task, I finally gave in and tried my hand using VBScript and the Windows Scripting Host. In four hours I had the basics of a script going.

My simple VBScript file does the following:

• Prompt the user for a PowerPoint file and target directory
• Launch PowerPoint and Word in the background
• Loop through every slide, inspecting any OLE objects found within to see if it was created by ChemDraw or IsisBase Sketch, two common chemistry editor tools
• If any such molecules are found, the following happens:
  • All of the “shapes” on the slide are copied to the clipboard
  • A new Word document is created with “Slide (1..n)” appended to the file name
  • The clipboard contents are pasted into the document

Sure, there are plenty of warts to work out, but I was happy to be able to run my script and watch as it magically consumed huge slide decks and spit out Word documents containing the exact data I was looking for. Windows Scripting Host is a powerful tool, as good as, if not better than AppleScript.

[Update: Added epilogue singing the joys of Windows Scripting Host]