Paper Jammed » Software

A couple of AppleScript droplets to tweak EXIF timestamps

Tad — Tue, 15 Feb 2011 04:52:45 +0000

Most of the time I don’t really bother with the timestamp information that my camera embeds in each digital photo. In fact, I can’t remember the last time I checked to see if the clock was right.

Scanned photographs are an entirely different brew. They typically represent events from the distant past, and scanner software EXIF data is hit or miss.

I looked for commercial software to handle a few special cases of EXIF data troubles, but came up empty handed. So I wrote a few useful AppleScript droplets that do these tasks quite nicely, and I will share them here.

Warning!

These scripts use jhead to manipulate and rewrite your JPEG files!

Don’t be a fool. Experiment first with a safe set of throwaway JPEGs. And never use these tools on original files; always keep a backup.

Prerequisite

All of these scripts depend on a fine piece of free software called jhead written by Matthias Wandel.

Installation is not difficult, but it does involve the command line.

Go to the jhead site
Scroll down to the Releases section and look for Pre-built OS-X Intel executable
Right click on the jhead link on that row and choose Save Linked File to “Downloads”

At this point, I found that jhead was saved as jhead.txt. Oh well. We needed to do some command-line magic anyway.

Open a terminal window and enter the following:

1
2
3
4

$ cd ~/Downloads
$ mv jhead.txt jhead
$ chmod 777 jhead
$ sudo mv jhead /usr/local/bin

These lines do the following:

Lines 1 and 2 navigate to the Downloads directory and remove the “.txt” from the name
Line 3 makes the file executable by everyone on your Mac
Line 4 places the file in a public area where everyone on your Mac can see it (you will be prompted for your password)

Stripping All EXIF Data

Sometimes I receive files that have corrupt EXIF data. I had a large quantity of scanned files in my collection that claimed to be scanned some time in 2038, while others insisted that they had been around since 1901. Neither situation is good, and I found that standard EXIF editing tools may fail to change these corrupt EXIF sections.

The answer is to blow away the EXIF data.

Download: Strip EXIF

This zip file contains a compiled AppleScript application. You can unzip it and place the application on your desktop. Safari will probably unzip it for you when you click the link.

To be safe, open AppleScript Editor and use it to open the Strip EXIF app to see its magic.

Now you can drop any number of JPEG files onto the Strip EXIF app and it will kindly eviscerate each JPEG, removing all traces of EXIF data.

Adding Basic EXIF Data to a vanilla JPEG

Some tools create JPEG files without EXIF date and time information within. This is typically the hallmark of photo manipulation software and dodgy scanner software. And if you happened to use the Strip EXIF app to rip out a bad EXIF block, then you will want to replace it with a proper data block so that you can still use camera date and timestamps.

Download: Add Basic EXIF

Again, unzip the file, place the app on your desktop, and then drop any number of JPEG files onto Add Basic EXIF.

The app will set the EXIF date to the file creation timestamp.

Spreading EXIF Timestamps

This is the real reason why I wrote these scripts. I couldn’t find a satisfactory tool on the market that would allow me to automatically spread out the shooting times for a series of images.

Why would anyone want to do this? Because some processes give you fifty JPEG files all with the exact same creation time and exact same shooting time. I like to use file renaming tools to incorporate the shooting time in the filename, so that files sort by chronological order. This doesn’t work if all of the timestamps are the same.

So I wrote a little app that adjusts the first photo by one minute, the second by two minutes, and so on. If there are fifty photos, then the last one will have its shooting time adjusted by fifty minutes.

The result is a series of photos/scans that have different timestamps.

Download: Spread EXIF Timestamps

Again, please look at the short program before you run it.

Closing Thoughts

I hope that my favorite tool implements these tricks soon (A Better Finder Attributes, I’m looking at you!), but until then, I will be dropping my files onto these three little droplets.

The jhead tool is so versatile that I will probably end up with a whole slew of similar droplets that will do all kinds of spiffy stuff. Nevertheless, I would rather the commercial products already provided these features. Not everyone likes dipping into AppleScript and the command line!

Why must basic copy/paste operations always result in “Ransom Note” text?

Tad — Tue, 01 Feb 2011 02:06:51 +0000

I estimate that in any given day I must use clipboard copy and paste operations a few dozen times—and if I’m working with multiple documents, such as when transcribing information from a spreadsheet to a Word document, that figure becomes more like a few hundred times a day.

But in all of those times, I almost never want my computer to paste “ransom note” text. Ninety nine times out of a hundred, I want clean text to be pasted, neatly blending with its surroundings. In other words, paste it as if I were typing it at the keyboard.

So why does Apple think we always want to paste styled text? And why does Microsoft think we always want our spreadsheet to suddenly be filled with somebody else’s 24 point Comic Sans letters instead of a nice unformatted date and time?

Here are my workarounds for this problem. Maybe someone can offer a few other workarounds.

Windows

The easiest way to pasting unformatted content in Windows is to use a free program called Pure Text by a gentleman named Steve Miller. Download PureText 2.0 for all modern versions of Windows.

This app doesn’t have an installer; it runs from wherever it is. However, since it does provide the ability to launch when Windows starts, I recommend you first drag the small executable file to a nice permanent home somewhere on your hard drive (I usually create a C:\Program Files\PureText folder).

Now launch the app. You will see a small PT icon in your system tray.

All you need to do for an unformatted non-ransom-note paste action is to hit Windows+V. In other words, instead of Control+V, hit the Microsoft flag key and V. You will hear a short cowbell sound as the paste action is performed.

Macintosh

Amazingly, there doesn’t seem to be such a free simple little tool available for Macintosh.

In some applications, such as Pages, the four-key combination of Option+Shift+Command+V will perform an unformatted Paste operation.

If you don’t mind spending a bit of money, programs such as Clean Text for Mac and SmartWrap 2 provide this functionality at a price: both are about twenty bucks. Fortunately, both products provide much more functionality than just stripping formatting from the clipboard.

If you want to roll your own, the best I could come up with is to use a keyboard macro application to assign a tiny script to the desired keys—this option works fine if you already happen to have a keyboard macro application installed. I use Keyboard Maestro, though this method should work just as well with other macro tools.

The essence of the “strip formatting” operation lies in this very short shell script:

1 2	#!/bin/sh pbpaste \| pbcopy

What this does is it simply pipes the output of a command-line “read clipboard” action into a command-line “copy” action. This is a text-level operation, so no funky formatting makes it through. Now, the clipboard contains only pure unformatted text.

The trick is to make your keyboard macro program execute this when you hit the key combination of your choice. Maybe I’ll find inspiration some day to learn how to make a small application like PureText for the Mac. When I do, I’ll post it here.

There are other options available for Macintosh. If you are ambitious, you can turn the problem around and Create a ‘copy as plain text’ service.

Why do computers do this?

When you copy text, the computer puts it on the clipboard in many different styles, referred to as flavors. Typical flavors might be “Rich Text”, “Unformatted Text” or “Image Data”. When you then perform a paste operation somewhere else, the receiving application sifts through the available flavors of your clipboard data and selects the one it likes the most. The application will usually pick the fanciest usable flavor available.

Usually, this process works fairly well. When you copy a selection of cells in an Excel spreadsheet, it might paste into a Word document as a proper Word table, but if you paste into an email application, you might get an image of the cells, and if you paste into a text editor such as Notepad, you will see just the numbers and words.

Likewise, a copied Visio diagram might paste into Word as an embedded editable Visio object, while a paste into Photoshop would result in a flat image, like a screenshot.

Final Thoughts

People have been swearing at the clipboard for years. The typical lowest common denominator is to paste the text into Notepad and the copy from Notepad. Even for this, Mac comes up short: the standard text editor, TextEdit, simply pastes in your ransom note text with formatting and all.

If you use Windows, go right ahead and install PureText. You won’t regret it.

If you use Macintosh, make it easy on yourself and invest in one of the payware applications. Think of how many times you have wanted to paste clean text, as if you had typed it in from the keyboard.

Get it while it lasts—Microsoft’s easy way to lock down a shared computer

Tad — Tue, 02 Nov 2010 01:56:26 +0000

Do you have a shared computer somewhere in your life? A computer that anyone and everyone uses in order to hop online to do a quick web search or to print a document?

I have been dealing with situations like this for years, working with computers in a small school and at a nonprofit volunteer organization, shared by many. It seems that whenever I turn on any of these machines, the background is set to something ugly, the screen resolution is weird, there is some cute animated mouse cursor, and someone has left their most intimate secrets in a document on the desktop.

Microsoft Steady State solves all of these issues by providing a means of creating a golden configuration that is restored to absolute perfection the next time the machine is rebooted. But download it before the end of the year, when it will be pulled by Microsoft!

Steady State Magic

This free product gives you the ability to configure accounts on your XP or Vista machine with several fine-level access controls. For example, you can prevent users from changing screen settings or prevent them from writing to anywhere other than their personal “Documents and Settings” directory.

But by far the coolest feature is the ability to turn off hard drive writes altogether. When you do this, Windows slips a layer between the OS and the physical hard drive that intercepts and tracks all hard drive activity during a session. During the session, the user can browse the web, create documents, install programs, whatever…but when the machine reboots, the cached list of hard drive changes is discarded completely: the hard drive is restored to the way it looked before the user booted the machine.

What can you use this for?

There are many places where a completely protected machine would be of great use…

A shared computer in a public area, like a hotel lobby
A home computer that is used by the kids and the cat and the dog
Computers in a school or library setting
Shared computers in a setting where many different workers use the same computer

Anything to worry about?

All of your users must remember that everything must be saved to a USB stick before reboot. Steady State warns you of this every time you reboot the machine.
There are some annoyances that might happen, such as that silly “Desktop Cleanup Wizard” popping up every single day because it thinks it hasn’t been run in five months, or “New Programs Installed” balloons that come up every single day because, again, the machine is restored totally to day-one upon reboot.
Microsoft is killing the product at the end of the year. Now it will likely remain functional for XP and Vista, but they are not upgrading it for Windows 7. But this is too cool a product not to try out. In theory, you could create a steady state machine today and keep booting today’s version of Windows XP for the next five years.

Additional Features

With the hard drive protection enabled, you can add programs at any time from an administrator account. When you shut down, Steady State will ask you if you want to commit your changed hard drive data to the Steady State disk image.

Even without the hard drive protection enabled, you have plenty of security constraints you can enable for other users to keep them from installing their favorite annoying toolbar and blinking mouse cursor. Think of this as a poor-man’s version of the domain policy tool used in enterprise environments.

More Information

Microsoft Steady State. How to remotely remove and retain changes on lab computers.
Defending the C disk with SteadyState from Microsoft
Alternatives to Steady State for Windows 7: Creating a Steady State by Using Microsoft Technologies
See Episode #129 of Steve Gibson’s Security Now podcast: Security Now! Episode Archive

Conclusion

I did not understand just how slick a tool this is until I installed it on a spare machine. It took about fifteen minutes to configure things right, but that machine has been running for the past few weeks with the locked-down golden configuration. Whenever it reboots, it looks exactly as it did when I installed Steady State.

Give it a try before it’s too late!

Sort out those disorganized thoughts with a Mind Map

Tad — Fri, 28 May 2010 01:35:37 +0000

You know the feeling: you are involved in some intractable problem that has all kinds of weird angles and you just can’t get your head around it—perhaps you feel like you are inspecting an elephant, one square inch at a time, or maybe you simply feel like you are herding cats.

There are plenty of different ways to catalog loosely associated knowledge of varying complexity—a few months back I discussed using a wiki for this—but some problems just don’t need that level of complexity and depth.

Some problems are more suited to random scribblings on a whiteboard, and that is where mind mapping software comes in.

What’s a Mind Map?

Imagine you and your family are going to fly to Rio de Janeiro this summer to visit distant relatives and you realize that there are about a thousand things to do in preparation but you just can’t sort it all out.

You know that you need passports and you need to verify that everyone’s visa is still valid. There is the monumental task of deciding what to pack. You might want to make a checklist of places you want to visit. And you want to go hang gliding down to the beach, but there’s something nagging at you about whether or not your health insurance would cover a broken leg in a foreign land.

The problem is that it is difficult to keep the whole thing in your mind—if you concentrate on the luggage, you forget about the international driver’s license.

Mind maps allow you to visualize the whole thing at once, and you can slide stuff around and get it looking nice and pretty.

This is a relatively simple start at a mind map that represents the vacation. These maps are typically read from the top right going clockwise, though many have no specific sequence. In fact, you can do pretty much what you like with a mind map as long as it works for you.

Just Another Outliner?

At first glance these tools look like glorified outlining applications such as OmniOutliner, and they do serve admirably in this respect, but they are so much more. An outline gives you a very easy way to organize topics and thoughts, adding annotations and such along the way, but it isn’t nearly as easy to visualize and nonlinear concepts do not map well to an outline.

Consider the Rio Trip example above—you could put all of that information into an outline, but it would not be nearly as easy to process mentally.

And these mind maps look especially cool in presentations.

Slick Document Generation

Some of the commercial mind mapping products provide pretty good integration with Microsoft Office products.

Some time back, I needed to write up a set of style standards for Oracle’s PL/SQL programming language for our offshore team. Rather than just dive into Word and hope for the best, I used Mind Manager Pro from Mindjet to make the following map:

This tool supported attaching rich text to nodes in the map, so I used these notes to handle the actual code examples.

I was then able to put together a nice Word template that matched our corporate documents and I clicked the Export to MS Word button and had an instant document:

I have also used mind maps to auto-generate PowerPoint slide decks as well. These days I use Mind Manager on a daily basis at the office; it makes a great difference when I am trying to grasp complex topics with lots of strange dangly bits hanging off of the edges.

Nothing New

Mind maps have been around for a long time. A quick search of the ‘Net will show you that mind mapping software is quite plentiful and mature. There are good free products for PC and Mac available and there are many commercial products that take mind mapping a step further, often integrating with Microsoft Office.

Take a look at what Wikipedia has to say about mind mapping. Here’s a post from 43 Folders on the same topic. Peter Russell has useful information as well about them.

I learned about mind mapping from a friend at work who had been using them for years. After seeing him make some quick notes during a meeting, I was sold.

Closing Thoughts

I made the Rio de Janeiro example using XMind, the free “lite” version of XMindPro. This application is pretty full featured for a free basic version—the Pro version adds enterprise features such as import/export and collaboration.

There are XMind versions for Mac, PC, and Linux.

If you are looking for more, all of the commercial products offer a trial period. I use MindJet’s Mind Manager. Be warned, these tools are expensive, just like buying MS Office, but you might just find that they more than make up for their cost with your newly found productivity.

Some problems are more suited to random scribblings on a whiteboard, and that is where mind mapping software comes in.

A short time back I was asked to serve on a project team that is involved in managing a stream of data going to dozens of downstream systems. Not only is the project massive, but each of these downstream products has its own project team and politics to deal with.

How do you even begin to learn about forty different computer systems? What do you do with all of the odd little tidbits of information that keep flowing in from all sides?

New life for an old PC—no geek card required

Tad — Thu, 06 May 2010 01:52:22 +0000

Do you still have an old machine kicking around in the basement or the back room, long forgotten?
For no cost and almost zero effort, you can set it up as a dedicated network appliance, using one of the many turnkey products from the open-source TurnKey Linux project.

I’m serious. You don’t need to know anything at all about Linux to use one of these. Just download the image, install, and you suddenly have a full featured NAS file server, or you might have a database or a source code repository.

Last year I wrote an article on how to set up a NAS device using Ubuntu Linux. I have been a fan of Ubuntu since the start because it is a very easy distribution to install and configure. The down-side of using Linux has always been the fairly steep learning curve. Before you can get around to using the server, you need to get down in the weeds with configuration files and other stuff.

TurnKey Linux changes all of that.

Painless Installation

A few weeks back, I was setting up an aging PC as a standalone wiki server for a small office—this machine was going to provide a place for the office staff to document their procedures, how-tos, and other things.

I was about to set up an Ubuntu server, as I have done before many times, and install MoinMoin, like I did some months back. I remembered that it was a bit of a pain to get everything tweaked just right, so I did a quick check to see what kind of standalone wiki options were available online.

This is how I found TurnKey Linux. This project is all about single-purpose preconfigured Ubuntu server images.

One of those preconfigured images happens to be a MediaWiki appliance—the wiki engine behind Wikipedia—and I was in business.

The installation took about fifteen minutes, with very little user interaction. I answered a few basic questions and the installer took over from there. As soon as the install was done, the machine rebooted and displayed a message on the monitor with the IP addresses where you can browse to from any other machine.

Full Featured

The work that has gone in to these appliances is amazing. In fifteen minutes I had installed a complex configuration that has the Apache, PHP, MySQL, MediaWiki core, as well as maintenance utilities such as a neat tool that provides a Flash-based pure-AJAX-based SSH command line in a remote browser (i.e. your browser becomes a terminal). Even someone with Linux experience would have to spend quite a bit of time fiddling around with different packages and configuration options in other to provide the same functionality that TurnKey gives you out of the box.

As with most open source projects, the documentation is about 80% complete, with deep detail in some areas, but leaving others fairly sparsely documented. But don’t let this deter you: in most cases users know how to use the product they are installing (e.g. MediaWiki) but don’t want the hassle of configuring it on Linux. That’s where TurnKey shines.

Some Examples

In minutes, you can set up a NAS device. If you want to try advanced content management in your office, try Joomla or Drupal.

If you are working on a small project team and want to protect your source code, try Redmine or Trac and do your bug tracking using Bugzilla.

And while you are at it, you can document your organization’s working practices using a wiki such as MoinMoin or MediaWiki.

Don’t forget to back it up!

As with any computer, you should include your new TurnKey appliance in your backup strategy. The nice thing is that you don’t really need to care at all about backing up Linux or the other software; just back up the data. I don’t need to back up my entire MediaWiki machine; I just need to back up the database and image files. If anything goes wrong, you can rebuild the TurnKey appliance from scratch in minutes and then restore your data.

To save yourself some pain, keep notes on any small tweaks you made to the configuration.

One Machine, One Purpose

These disk images share common Ubuntu underpinnings, but they are referred to as Appliances because they turn your PC into a purpose-built appliance.

This means that if you want a content management system and you also want a ticket management system, you will need two old computers—not a rare commodity these days.

Take a look at what they have to offer and give TurnKey a shot—specialized software used in corporate environments is now within reach of small offices at the right price.

A handful of sweet freebie tools to save the day

Tad — Wed, 17 Mar 2010 03:31:14 +0000

It so happens that my employer has made a most welcome decision to replace the aging creaky old Novell GroupWise mail software with Microsoft Outlook, joining the rest of the modern corporate world. Now, there is little love in my heart for GroupWise, but it does have one feature that the new Outlook configuration will lack: you can keep as many emails as you want, just like Gmail.

The problem is this: with Outlook we will be limited to 1000 messages in our in-box; sadly, many of us have tens of thousands of emails in our old GroupWise mail. Even after a fairly rigorous slash and burn mission, hacking out all of the low hanging fruit, there will be many thousands remaining and I don’t want to lose that information. It might be useful to search and find how I set up a Zebra bar code printer in 2003, no?

A bundle of different freeware glue tools came to my rescue. Read on to hear about the toolset that has made it so I can keep those messages for years to come.

Possible Solutions

Right out of the gate, I began looking for ways to migrate messages from one mail client to the other. Some apps have this built right in, and if not, there are scripts and utilities out there to do this; but I was hampered by a few key facts:

I have no control over the email clients and their configuration. Even if there is a menu option for exporting GroupWise messages from version 7.2, I’m stuck at 6.4 and cannot use that option.
GroupWise is a minor player in the email world. I’m not sure if Outlook would import from GroupWise, but I doubt it.
They are replacing the client in one shot. There will be no interim period where both GroupWise and Outlook will be available.
There is no getting around the hard limit of 1000 messages.
I don’t want to spend money on this.

With these constraints in mind, I immediately thought about PDF documents. I then considered the following questions:

How do I convert my email to PDF?
How can I do this automatically with thousands of emails?
Once I’m done, how do I search these documents?

Here’s what I did:

Conversion to PDF

The first part was easy. I downloaded one of the many free print-to-PDF products available.

I chose PDFCreator, because I am familiar with its use and I know that it does not munge the fonts.

Like many other PDF generation utilities, PDFCreator functions by providing a virtual printer to which any application can print. For example, to make a PDF of a web page, you use the Firefox Print menu and select PDFCreator from the drop-down list of available printers.

You are provided with a list of metadata fields that you can fill in, and these fields are used in the PDF generation.

Here’s what the PDFCreator screen looks like:

A word of caution: PDF Creator is free, but you must be careful to deselect their spammy toolbar options in two different places during the installation process. I don’t like software that comes with preselected toolbars to install (even nice ones like Google’s) because I’m certain that 95% of the folks who actually install the toolbar would never have chosen to do so if it were unchecked by default.

Running Everything Automatically

This was the interesting bit. I work with Windows machines at work, so there was no AppleScript option available. So I did the next best thing: I used AutoIT.

I will warn you that AutoIT is pretty much the Windows analog of AppleScript, without the cutesy pseudo English syntax. In other words, you will need to roll up your sleeves and get your hands a little dirty in order to put together a decent AutoIT script.

The payoff comes when you finish your work and compile it into a tight executable that you can share with your friends, allowing them to automate some complex series of button clicks and copy/paste operations.

I walked through the manual process of exporting an email to PDF and listed each action:

Get the date, sender, and subject
Create a filename based on date + sender + subject
Launch the Print dialog
Select PDFCreator
Fill in the Document Title, Creation Date, and Subject in the PDFCreator dialog
Fill in the full file path in the Save dialog

In addition, I wanted to make the script a little better by adding the following:

Check that user has PDFCreator installed
Verify that GroupWise is running and that the user has selected one or more messages
Prompt the user for a target directory before processing the messages
Sanitize the filenames by replacing illegal characters with underscores and truncating to meet maximum filename and path length in Windows
Skip over files that have already been generated, quickly, so that one doesn’t need to worry about accidentally selecting messages that were already printed

There were other adjustments needed, but the process was the same: run the script, hit a problem, tweak the script a little to address the problem, and repeat.

Here’s a little bit of the AutoIT script:

You can see that it is a bit more intense than AppleScript, but remember that the full script wasn’t written in one go. I had a little short ten-line script that I kept tweaking as small problems cropped up until I had adjusted things to my liking.

Note that this is a GUI macro language. The machine starts clicking and typing away right in front of you and you probably shouldn’t interfere until your script finishes.

As of this afternoon, I have generated around 4,000 PDF documents for my email messages.

Searching All of Those Documents

This was the easiest part. These days there is an excellent tool available for searching documents on your desktop: Google Desktop. This product indexes every useful file on your desktop and provides a full Google search with a quick double-tap of the key.

So you can enter a search like “Zebra bar code”

And the results look exactly like a Google web search, but it’s showing your desktop files. And you can see inline previews too.

Macintosh users can install Google Desktop as well, but all of these files should already be indexed and searchable by Spotlight.

Closing Thoughts

Whenever I reach for tools like this I feel a twinge of guilt—it’s outright hackery, isn’t it?

But there is a place for quick and dirty jobs in every workplace. I needed to get my files from one place to another, one time only. It just didn’t make sense to spend money or time on a more elegant solution.

Play around with each of these tools a little. Especially AutoIT—it’s a handy Swiss Army Knife to have at your disposal.

Don’t punish your family with stacks of photos!

Tad — Thu, 25 Feb 2010 04:18:47 +0000

A while back I had a rare opportunity indeed: I went on a business trip to India to visit our offshore team. We knew it was a once in a lifetime trip, so four of us took some vacation days and paid our own way on a side trip to some of the great cities of India after the business was done. When we finally sat down to pool our collection on layover in Frankfurt, there were over 1,500 photos.

What do you do with 1,500 photographs?

In hope of sparing some folks hours of boredom I’d like to share my ideas on this topic here.

The Endless Stack of Photos

We have all been there. A friend or family member brandishes a stack of photos, saying “Let me show you photos of our trip to Ecuador…” (oh no… here it comes…). At this point, you reach for the photographs, but they hold the stack out of reach. They then turn each one over slowly, telling a long tale about every single image. “Oh, look at this monkey, it was so cute when he stole the candy out of little Billy’s hand and spit it into Aunt Sally’s hair.” and so on and so on.

You begin to look at the size of the stack and estimate how long this process will take.

Everyone has been on the receiving end of this treatment, but have you ever been the perpetrator?

It’s an easy trap to fall in to. To be honest, when you are showing photos to a friend, each image brings back a wave of pleasant memories and it is tempting to bask in the enjoyment of the memory, talking about how you felt at the time, as your friend’s eyes begin to aquire a glossy sheen.

Don’t Be That Person

Remember how it felt the last time you endured a four hour photo flipping marathon and have pity on those around you.

Here’s my strategy for pleasant photo sharing:

Pare down the photo collection. Substantially.
Create an attractive photo album with the finest photos of the lot.
Hand the album to your friend and let them turn the pages.

Back to those 1,500 photos from India…

Here is one of the pages from the album I made from that trip.

As you can see, the end result is fairly simplistic, with a few very nice photos.

Paring Down the Stack

Even the most avid photographer understands that nobody wants to see a thousand photos. And don’t think that just because you made a slideshow with music instead of printing the photos that you are exempt. You can afford to cut the number down quite a bit.

Consider that a typical Hollywood motion picture contains less than 10 percent of the total footage filmed. Stanley Kubrick, ever the perfectionist, took this to the extreme with shooting ratios around 100 to 1. Following the analogy, in photography, it is quite reasonable to take dozens of photos for every single picture that you might share to others.

The real trick is deciding exactly how far to go with the selection process.

In my experience, you can weed out the bad photos for hours, and when you think the job is done, you can still go back and toss out a few dozen more.

I filter my photos in three major phases, using the five-star rating tool of my photo library software to help keep things in order. I personally use iPhoto, but any other good photo library suite should offer ratings and smart folders.

Phase 1: Removing obviously bad photos

This is a very quick pass through the whole collection. I start by selecting everything and marking all photos with a neutral rating of three stars.

I then find any photos that are underexposed or are blurry and give them one single star. Along the way, any photos that obviously have no useful content get the same treatment.

Phase 2: Identifying decent photos

I use a smart folder to show all photos with three stars or greater. This hides all of the junk from the first pass.

Now, I go through each photo and give it a deeper look. I sort them into three different stacks, giving two stars to anything that has useless or boring content and giving four stars to photos that I think are worth showing to people. Photos that don’t fit either description retain the neutral three-star rating. These are often repeats of the one good photo I tossed in the four-star stack.

Phase 3: Identifying the best of the best

I use a new smart folder to show only photos with four stars or better.

This is the hardest part. I go through the photos and try to find the absolutely best photo that expresses each experience or thought.

If the photos are of places, then it makes sense to check that you have at least one photo of each important place you visited.
If these are shapshots of friends and family, then you probably should verify that each person shows up in at least one of the pictures.

I often have a difficult time working through photo collections from visits to my wife’s family in Brazil: there are hundreds of people in the pictures and I often have doubts over who is family and who isn’t. Fortunately, my wife sits patiently with me and helps at this stage.

Look again at the photos of the elephant ride and the snake charmer. I probably have two dozen different shots of the snake charmers, while the elephant shot was a single dodgy photo taken by the tour guide. I was able to pick the very best snake charmer photo, but I had little choice with the other—there was no way I was going to omit a picture of me on an elephant so I used it. These are the kinds of tradeoffs we are dealing with.

You can see in this screenshot how I flagged most of the photos as three stars (I have already hidden all of the one-star dreck). There is one photo that has four stars, while the one next to it had an unappealing composition in my opinion, so it got two.

How far should you go?

I suppose this comes down to personal preference, but I like to keep things down to thirty or forty photos—from a starting point in the thousands. In a photo album, you can represent your whole trip in fifteen or twenty pages. This is far less intimidating than a big thick stack of photographs.

Of course, context is important. I will go out on a limb here and say that a new baby can be shown to co-workers in five photos.

In the end, I create a smart folder that shows only photos with five stars. And boy do they look good!

Tweak the best photos

My favorite tool of all for tweaking photos is the crop tool. A good crop can dramatically change the composition of a shot while still retaining the purity of the photo.

I will also straighten any slanting horizons and possibly fix funky light balance at this point. The tool set provided in iPhoto is quite adequate for these simple tasks.

Now create a slick photo album using any of the great tools available online.

Make the Photo Album

Again, I like using iPhoto. It allows you to easily create fancy albums using templates and so forth. Once you are done, you can buy a finished album with a few clicks.

Once you have your short list of photos, use a five-star smart folder as the source for the photo album. You can then spend a pleasant evening or two playing around with the layouts and composition and adding captions to your photos.

Create a smart folder that shows only five-star photos
Create a new photo album based on that smart folder
Choose a pleasing layout
Add your photos in varying page styles to the book
Write some informative and/or witty captions for the photos

This is a screenshot of the iPhoto application, with the A-list photos along the top and the snake charmer page in the editing window.

You can even make albums like this online, without any editing software whatsoever…

This is a similar photo album creation tool that is run completely from the Winkflash website.

Now, finish picking out the features you want on your album (e.g. cover style) and place your order.
In my opinion, these albums come with the optimal number of pages (usually 20). Any more pages could make it tedious and boring. Fit your vacation into those 20 pages.

These books usually cost around forty bucks, but they are worth every penny.

The Finished Product

When everything is done, you will have a beautiful printed photo album that looks like you bought it at a book store.

I have seen photo albums from three different outfits up close.

MyPublisher

I have a couple of albums from these guys and they are near perfect. The pages look like thick magazine pages, with magazine-quality photos. I found that my white-on-black text bled a little.
Note that the leather used on the leather-bound books is paper thin.

These folks are always sending me coupons for 40% off, so it seems that you really don’t ever have to pay full price for their wares.

Apple iPhoto

These are identical to the MyPublisher books. Indeed, at one time iPhoto could send its output to either MyPublisher or Apple. I heard a rumor that MyPublisher did the books for Apple at one point.

The only down side to the Apple books is that you pay high shipping costs. Otherwise, the books are perfect.

Winkflash

A friend shared one of these with me. I happened to have my India album nearby, so we compared them. On the one hand, Winkflash is much cheaper; however, the image quality is not nearly as nice as the other books. Perhaps I was looking at a lower-end book from them, but the ink dots were a little coarse for my taste.

Summary

By putting a little effort in aggressive photo selection, basic image tweaks, and then taking advantage of the many photo book tools out there you can create a beautiful album that is a pleasure to leaf through.

These books have been available for several years; even so, whenever I hand one to a friend, they page through it, transfixed. People really love these albums and they actually enjoy looking through them.

Oh, and they make great gift ideas too!

What does this topic have to do with reducing paper in our lives? Believe me, printing one of these books is so much neater and cleaner than printing hundreds of loose photos. And you will enjoy them more.

Automate ScanSnap OCR process on your Mac with AppleScript (Snow Leopard Edition)

Tad — Tue, 05 Jan 2010 01:51:52 +0000

Some time back I published an AppleScript that allows one to automatically run OCR in the background on scanned files generated by your Fujitsu ScanSnap, while you to continue scanning more files. ScanSnap owners should all be familiar with this: the out-of-the-box configuration of the ScanSnap Manager and Abbyy Finereader force the scan and OCR stages to run in lockstep: scan 1…OCR 1…scan 2…OCR 2… and so on. This script allowed you to scan regardless of the OCR processing going on.

As it turns out, my original script does not work in Snow Leopard, and I promised that I would one day clean up and publish my new and improved version.

Chris posted a comment today as a gentle reminder, so here is the new and improved version without further delay…

The Details

Unfortunately, Snow Leopard came around and caused some indigestion. For starters, the ScanSnap Manager didn’t work correctly and Abbyy Finereader would not process anything made by the ScanSnap. A couple of months later they got everything straightened out and delivered new versions of each product.

The new version of the Abbyy Finereader product does not play well with my original script.

Since I cannot do without this important functionality, I rolled up my sleeves and rewrote most of the script. The new version works in Snow Leopard quite nicely with one small annoyance: you really don’t want to try to use the machine for anything other than scanning or OCR while it is going because the new Finereader version keeps bouncing the darned icon all the time it is running and that is quite annoying to watch.

Fortunately, I really don’t need to use my machine for anything else while it is chewing on the docs; I just wanted to be able to continue scanning at the same time!

Note: Before going forward, note that you will need to upgrade the ScanSnap Manager and Abbyy Finereader to the Snow Leopard versions first! Get the files here.

Here is a link to the new script…

And here’s the code itself:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234

(*

NOTE: This script was written for Snow Leopard. It may work
on Leopard, but I never tried it.

This is a folder listener script that will act as a queue, receiving
PDF files from the ScanSnap scanner and feeding them, one by one, to
the Abbyy FineReader OCR software.

This allows you to keep scanning while the OCR job runs in the background
on all of the unprocessed files.

Why do we want to do this?

The ScanSnap Manager software does not support this by default, so
when you scan in a file, it sends it to FineReader for OCR. You then
must wait until FineReader finishes its work before scanning in another
document.

This script allows you to keep scanning without waiting for OCR.

Installation:

o Copy this script to:

/Library/Scripts/Folder Action Scripts

You may have to create the "Folder Action Scripts" folder.

o Open a Finder window and navigate to the parent folder
of the scanned documents folder.

o Right click (control-click) the scanned documents folder and
choose:

Folder Actions Setup...

o At this point if folder actions are not enabled, you will
likely have to enable them and add the script manually.
- check "Enable Folder Actions"
- Use the "+" buttons on the left and right sides to add the
scan folder and then this script.

o Otherwise, a list of scripts will come up. Choose this script
from the "Choose a Script to Attach" dialog.

o Close all windows.

Copyright (C) 2010 Tad Harrison
*)
property ocrFileSuffix : " processed by FineReader.pdf"
property ocrApplicationName : "Scan to Searchable PDF"
property ocrApplicationWindow : "Converting the document"
property ocrLockFileName : "OCR in Progress"
on adding folder items to this_folder after receiving added_items
set lockFilePath to (POSIX path of (path to desktop folder as text)) & ocrLockFileName
try
logEvent("=== Run OCR on New Folder Items ===")
-- Test for lockfile; exit if lockfile exists
tell application "System Events" to set lockFileExists to exists file lockFilePath
if lockFileExists then
logEvent("Other script running. Exiting...")
return
else
do shell script "/usr/bin/touch \"" & lockFilePath & "\""
end if
-- Main loop
set moreWorkToDo to true
repeat while moreWorkToDo
set aFile to getNextFile(this_folder)
if not aFile = "" then
ocrFile(aFile)
else
set moreWorkToDo to false
end if
end repeat
logEvent("No more work.")
exitApp(ocrApplicationName)
on error errorStr number errNum
display dialog "Error " & errNum & " while running OCR: " & errorStr
set my isRunning to false
end try
-- Get rid of the lockfile, ignoring any errors
try
do shell script "/bin/rm \"" & lockFilePath & "\""
end try
end adding folder items to
(*
Name: ocrFile
Description: Runs OCR on the next un-OCR'd file
Parameters:
aFile - the file to be OCR'd
*)
on ocrFile(aFile)
set posixFilePath to POSIX path of aFile
set posixOcrFilePath to getPosixOcrFilePath(posixFilePath)
logEvent("OCR: " & posixFilePath)
tell application ocrApplicationName to open aFile
--
-- Now sit in a loop checking once per second for the OCR file
-- Give up after five minutes
--
with timeout of 300 seconds
set ocrFileExists to false
repeat until ocrFileExists
set ocrFileExists to posixFileExists(posixOcrFilePath)
if ocrFileExists then
logEvent("OCR file generated.")
-- Wait 5 even if the file was found, to let things settle
delay 5
else
-- Wait a second before checking again
delay 1
end if
end repeat
end timeout
end ocrFile
(*
Name: appIsRunning
Description: Determines if a particular application is running.
Parameters:
appName - the name of the application to be tested
Returns: True if the application is running; otherwise False
*)
on appIsRunning(appName)
tell application "System Events" to (name of processes) contains appName
end appIsRunning
(*
Name: posixFileExists
Description: Determines if a particular file exists.
Parameters:
posixFilePath - the POSIX path to the file
Returns: True if the file exists; otherwise False
*)
on posixFileExists(posixFilePath)
tell application "System Events" to exists file posixFilePath
end posixFileExists
(*
Name: exitApp
Description: Exits the specified app if it is running.
Parameters:
appName - the application name
*)
on exitApp(appName)
if appIsRunning(appName) then
tell application appName to quit
end if
end exitApp
(*
Name: getPosixOcrFilePath
Description: Gets the OCR output filename for a given input filename.
Parameters:
posixFilePath - the full path to the source file
Return: the POSIX path of the OCR output file
*)
on getPosixOcrFilePath(posixFilePath)
set posixBaseName to do shell script ¬
"filename=" & quoted form of posixFilePath & "; echo ${filename%\\.*}"
set posixOcrFilePath to posixBaseName & ocrFileSuffix
return posixOcrFilePath
end getPosixOcrFilePath
(*
Name: getNextFile
Description: Finds the next unprocessed ScanSnap PDF
Return: the file or ""
*)
on getNextFile(aFolder)
logEvent("Getting next file...")
set masterFileList to list folder aFolder ¬
without invisibles
set posixPath to POSIX path of aFolder
repeat with i from 1 to count masterFileList
set fileName to item i of masterFileList
set posixFilePath to posixPath & fileName
log posixFilePath
--
-- Construct a FineReader file name from our file
--
set posixOcrFilePath to getPosixOcrFilePath(posixFilePath)
--
-- See if the FineReader file we constructed exists
--
set ocrFileExists to posixFileExists(posixOcrFilePath)
tell me to set fileCreator to getSpotlightInfo for "kMDItemCreator" from posixFilePath
log ("Creator: " & fileCreator)
if not ocrFileExists and fileCreator = "ScanSnap Manager" then
return POSIX file posixFilePath
end if
end repeat
return ""
end getNextFile
(*
Name: getSpotlightInfo
Description: Gets a named attribute from metadata for a specific file.
Parameters:
for myattribute - the name of the attribute
from myfile - the name of the file
Returns: the attribute value or "" if none found
*)
on getSpotlightInfo for myattribute from myfile
try
set this_kMDItemResult to ""

tell application "Finder"
set this_item to myfile as string
set this_item to POSIX path of this_item
set this_kMDItem to myattribute
set theResult to words of (do shell script "/usr/bin/mdls -name " & this_kMDItem & " -raw -nullMarker None " & quoted form of this_item)
log "Result: " & theResult as string
repeat with j from 1 to number of items in theResult
set this_kMDItemResult to this_kMDItemResult & item j of theResult as string
if j < number of items in theResult then
set this_kMDItemResult to this_kMDItemResult & " "
end if
end repeat
end tell
on error
set this_kMDItemResult to ""
end try
return this_kMDItemResult
end getSpotlightInfo
(*
Name: logEvent
Description: Write an event to an event log
Parameters:
themessage - the message to write to the log
*)
on logEvent(themessage)
set theLine to (do shell script ¬
"date +'%Y-%m-%d %H:%M:%S'" as string) ¬
& " " & themessage
do shell script "echo " & theLine & ¬
" >> ~/Library/Logs/AppleScript-events.log"
end logEvent

Installation

Use the Script Editor to save this script as Run OCR on New Folder Items under User Home/Library/Scripts/Folder Action Scripts
You may have to create the Folder Action Scripts folder.
Now open a Finder window and navigate to the parent folder of your scanned documents folder.
Right click (control-click) the scanned documents folder and choose Folder Actions Setup…
At this point if folder actions are not enabled, you will likely have to enable them and add the script manually.
- Check Enable Folder Actions
- Use the “+” buttons on the left and right sides to add the scan folder and then this script.
Otherwise, a list of scripts will come up. Choose this script from the Choose a Script to Attach dialog.
Close all windows.

That’s it! The script will be invoked automatically every time a new file appears in your scanned documents folder.

Please let me know if you have any ideas that can improve this script. I’m not an AppleScript guru, so someone might just know how to keep that annoying Finereader icon from jumping.

Keeping your secrets to yourself—old changes lingering in your PDF files

Tad — Tue, 24 Nov 2009 04:46:58 +0000

A few months ago I wrote an article that touched upon the problems inherent in attempts to sanitize documents before sending them to the enemy—perhaps to remove competitor’s names or trade secrets.

I was reading a post on a board I frequent where a person was describing exactly this kind of activity—removing sensitive information from PDF documents. Several suggestions were made, but one individual suggested opening the file in Acrobat Pro and replacing the sensitive text with good old Lorem Ipsum.

It was at that moment that I recalled a peculiar feature of the PDF file format: it is designed to support nondestructive updates, allowing people to make vast changes to a PDF document while still retaining the original document, fully intact. I did a few experiments and was surprised with the results.

A Brief Note on the PDF File Format

For the geeky types among us, one place to begin is this article:

Portable Document Format: An Introduction for Programmers

The key points to get out of the article is this: A PDF document is comprised of several distinct sections, a Header, a Body, an “xref” Table, and a Trailer. At the very end of the file you will find the character sequence %%EOF

The PDF standard was designed to allow multiple updates to a document, while retaining the original version. This is accomplished by appending anything new to the end of the document, after the original EOF tag. The document will now have two EOF tags: one indicating where the original document ended, and a new EOF tag indicating where the new changes end.

If we wish to revert PDF changes, it should be a simple matter of opening the PDF file in a binary editor, searching for the first EOF tag, and deleting everything following.

A Simple Experiment

Let’s start with a proper secret document containing missile plans…

Suppose we want to obscure some special information in paragraph 37. We can open the file in Acrobat Professional and use its text editing features to swap in the venerable Lorem Ipsum text.

Here’s what it looks like after the switch:

You can see here that the first seven lines of text starting on paragraph 37 have been replaced with appropriate unreadable text.

Now, open the new PDF file in a binary editor (since PDF files contain a mix of text and binary, the editor must be a binary editor).

Note the %%EOF character sequence embedded in the text. This is the first EOF tag, indicating where the original file ended. All we need to do is place the cursor to the right of the EOF and delete everything to the end of the file.

Once we have done so, it’s like magic:

The edits that replaced lines of paragraph 37 with gibberish have neatly been undone!

More Details

From the PDF Intro document linked earlier:

“The trailer, it turns out, plays an important role in the way PDF implements incremental updating. The key concept to understand here is that a PDF file is never overwritten, only added to. That goes for all portions of the PDF file – even the trailer itself, and the end-of-file marker. In other words, a multiply-updated PDF document may contain multiple trailers – and multiple end-of-file markers! (There may be numerous occurrences of %%EOF.) Each time the file is edited, an addendum is written to the tail of the file, consisting of the content objects that have changed, a new xref section, and a new trailer containing all the information that was in the previous trailer, as well as a /Prev key specifying the byte offset (from the beginning of the file) of the previous xref section. The cross-reference info will then be distributed across more than one xref section. To access all of the cross-references, the reader must walk the list of /Prev keys in all the trailers, in reverse order.

Space doesn’t permit a detailed exploration of updates here, but you can find several examples in Appendix A of the PDF 1.3 specification (available at http://partners.adobe.com/asn/developer).”

Summary

It is important to understand that the PDF standard allows for appended updates to files that leave the original document intact, regardless of how drastic the changes are. If you are intent on redacting text from PDF documents, do not depend on simply deleting the secrets using a PDF editor—you must use a proper redaction tool that addresses these issues correctly.

That said, I did some experimenting with a few utilities (Apple Preview, PDFpen, and Adobe Acrobat Pro) and found that some write the file from scratch each time, with no lingering cruft from former versions, while others respect the original intent of the PDF standard. This means that you can’t trust that older revisions are being retained in your file and you can’t trust that they aren’t.

Be conservative: use a redaction tool for secrecy and proper backups for versioning.

Snow Leopard Update for ScanSnap

Tad — Sat, 14 Nov 2009 04:36:58 +0000

This evening I opened my email and found a most welcome message: Fujitsu has released their patched version of the ScanSnap software for Snow Leopard.

[UPDATE: I spoke too soon—they only delivered half of the goods. See below.]

[UPDATE 2: Hurray! It's fixed! The birds are chirping and the sun is shining and life is good!]

When Snow Leopard came out back in August, I ordered my copy the first week and was so excited that I installed it the day it arrived. My joy was short-lived, however: the most important software package I use did not work!

I depend greatly on the OCR capabilities of the ABBYY FineReader software that comes with the ScanSnap scanners, and this was one of the many pieces of software that did not smoothly transition to Snow Leopard. I could scan documents, with limited functionality, but the OCR feature did not work.

Now that Fujitsu has released their official update, I will probably be installing Snow Leopard tomorrow evening. Now, do I do an upgrade or a full install? Hmmmm…

UPDATE

Well, they only delivered half of the goods.

I was reading through the seven-step process for updating the ScanSnap drivers and I arrived at step seven:

Step 7: The download for FineReader for ScanSnap update to Snow Leopard will be hosted by ABBYY but is not yet available. If you have already subscribed to be notified by Fujitsu regarding the Snow Leopard updates, an email will be sent to you when it is posted.

How displeasing. The only thing I really cared about was getting the OCR to work, and apparently Abbyy has not yet delivered their part (How hard can it be to update the part in your code that says “If the PDF metadata doesn’t match X then the document isn’t a ScanSnap doc” ?).

UPDATE 2

Instead of making us wait another month or two, ABBYY has delivered their patch in record time . I quickly installed the update and tested my ScanSnap script…

There are a few kinks to work out, but it is pretty clear that I will have my same old workflow AppleScript folder action up and running in short order. I will post the newer script as soon as I have it running properly.

And, as a side note, I had already purchased PDFpen from SmileOnMyMac as a backup plan. Their tool incorporates the OmniPage OCR engine, an engine that rivals that of ABBYY. My script was already running with PDFpen, but there were some issues with tables in documents that I forwarded on to SmileOnMyMac. One of these days I’ll post my scripts for PDFpen.