I like the playlist tools that iTunes provides, and I find it very convenient to create Smart Playlists or to create Genius lists. However, I want to keep my Master Library of music elsewhere, far from iTunes. Wouldn’t it be great if I could share these playlists on my home network in a completely nondenominational format? Wouldn’t it be nice if my Squeezebox server would have those same playlists available? And wouldn’t it be spectacular if those playlists would magically appear on a network drive whenever the lists change in iTunes?

Read on for details on how I accomplished this and to download the free utility I wrote to handle this task.

The Problem To Be Solved

In order to understand what this application does, you need to know what I was trying to accomplish.

I have a NAS drive on my network that contains my Master Library of mp3 files. The files on this drive are located under /media/music.

I have a Squeezebox server, running on an Ubuntu machine. This server is used by Squeezebox devices to play music anywhere in my house. It holds a copy of the Master Library, located under /srv/squeezebox/music

What I want: I want my Mac to magically push proper m3u playlists to the Squeezebox server—the Squeezebox server understands m3u playlists.

What I don’t want: Squeezebox supports direct access to iTunes. But I don’t want to do this because I don’t want to have to run Squeezebox on my Mac. Why did I bother setting up a dedicated server if I have to keep my Mac running?

Why is this challenging: Even if I could point Squeezebox to the bare iTunes database file, all of the file paths are wrong. My master library might have a song at …/artist/album_name/disc 1/, while iTunes might put it at …/Artist/Album Name/, omitting the disc 1 folder and reformatting names.

My Solution

I wrote a small command line application in Perl that supports the following:

• Paths to songs in Apple’s mysterious black box are converted to nice paths to the same songs in my golden Master Library.
• Even if a song name changes or a folder path changes, the original master copy is found.
• All playlists are generated in m3u format.
• Smart playlists and Genius playlists are supported.
• The generated playlists are automatically moved to either a local directory or a remote server, using scp.
• Network shares are automatically mounted at beginning of process and unmounted at end of process.
• You can provide a file path to be prepended to all playlist entries.
• Exits early if the iTunes library has not changed since the last run.
• Supports exclusion patterns in order to skip some playlists.
• A local Squeezebox server can be automatically pinged to cause it to refresh playlists.

Here is the full documentation for the Playlist Mapper application.

Basic Installation

Please note that this is a command line program. Some day I might figure out how to make a pretty GUI installer for it, but for now you have to get your hands a little dirty in order to install and configure.

Download the application here: plmapper

This is a Perl script that uses many standard utilities already present in a Snow Leopard OS X installation. You should not need anything other than the plmapper file itself.

Unzip the file and place plmapper in your home directory.

Open a Terminal window and make the script executable:

chmod u+x plmapper

Read the man page for the application by doing this:

./plmapper --man

There are many command line arguments, but you don’t need to worry about most of them. Create a file called plmapper.config in your home directory and put any arguments you need in that file, as name-value pairs.

Example:

itunes=/Users/Bozo/Music/iTunes Music Library.xml
dest=/Users/Bozo/Desktop
library=/Volumes/media/music

Once you have added all of the configuration settings, go ahead and run it:

./plmapper

You should see plenty of status information go by as each playlist is processed.

Making it Run Automatically

There are better ways of doing this, but I’m a command-line commando, so I simply added the following cron entry (see crontab):

0 22 * * * /Users/Me/bin/plmapper >/Users/Me/plmapper.log 2>&1

This causes the plmapper program to run once daily at 10pm, logging output to a file called plmapper.log in my home directory.

Of course, this can be tweaked to taste; it doesn’t matter if you run it hourly since the app skips any heavy lifting once it sees that the iTunes library file hasn’t been touched.

How It Works

The application performs the following basic steps:

• Mounts network shares using mount_smbfs.
• All music files in the external non-iTunes Master Library are listed in an internal data structure, sorted by file size.
  This allows us to quickly find a “short list” of possible candidate files that match a given source file. This works because mp3 files have fairly random sizes.
• The iTunes database XML file is processed using an XSLT transformation to generate a set of basic m3u playlist files. The xsltproc command line utility is used to run the transformation.
• All paths in the playlists are converted from escaped URI syntax (e.g. %20 for space) to regular UTF-8 characters.
• Each path is resolved to a single music file (i.e. an mp3 file) in the Master Library.
  This is done by obtaining the size of the iTunes version of the file and then looking this up in the internal list created in the first step.
• Once a short list of candidates is found, the first few thousand bytes of each file are compared until a match is found.The risks of choosing the wrong file are low: all that will happen is that a song might be switched in the playlist accidentally. As such, it isn’t worth scanning the full file to ensure they are exact matches.This algorithm works quite well, and can detect music files that have been renamed and moved around.

• The playlist files are now converted from utf8mac to utf8 using the iconv command line utility.This process collapses wide UTF-8 characters, consisting of a letter plus an accent add-on, into their single-letter equivalents. This process is known as Unicode Normalization, and this tool uses Normalization form C: Compatibility decompisitiion followed by canonical composition.
• Playlist files are now copied to a local or remote directory, either as direct file copy or via the scp utility.
• Optionally, a special call is made to a specified Squeezebox server to tell the server to refresh all playlists.
• Any mounted shares that were mounted by this process are unmounted via diskutil unmount.

Limitations

• Nested folders of playlists currently act funny. The lists will come over, but you may get two lists. You can use the exclude option to prevent this.
• Songs with non-ASCII characters in the file name might not be recognized by all players. Squeezebox has a problem with many special characters—I tested this by adding songs like Águas de Março.mp3 using Squeezebox’s own playlist editor, and when I reloaded the list the songs were not there. This is a bug in Squeezebox server.
• Occasionally the file matching algorithm might pick the wrong song. In an effort to speed up processing, the program only looks at the first few thousand bytes of files when comparing. It isn’t a big deal if the wrong file is added to a playlist, is it?

Closing Thoughts

As often happens, this program was written because I had an itch that needed to be scratched. Now that I went through all of the hassles, I hope others can benefit.

Through good fortune, I am working in a Macintosh environment. This means that all of the important tools were already present on my machine: perl, xsltproc; mount; iconv; scp; nc; and others. If I were writing this for a PC, I probably would have written the whole thing in Java—a more familiar language to me, but one that comes with its own configuration issues.

I plan on updating plmapper over time, and if anyone has suggestions, please let me know.

And if a Perl guru wants to tell me my Perl skillz are subpar, go right ahead—but be gentle, and let me know how to improve the app.

[Update]

Fixed a couple of bugs related to path and “qx” calls, thanks to Chris. Updated version here.

Scanned photographs are an entirely different brew. They typically represent events from the distant past, and scanner software EXIF data is hit or miss.

I looked for commercial software to handle a few special cases of EXIF data troubles, but came up empty handed. So I wrote a few useful AppleScript droplets that do these tasks quite nicely, and I will share them here.

Warning!

These scripts use jhead to manipulate and rewrite your JPEG files!

Don’t be a fool. Experiment first with a safe set of throwaway JPEGs. And never use these tools on original files; always keep a backup.

Prerequisite

All of these scripts depend on a fine piece of free software called jhead written by Matthias Wandel.

Installation is not difficult, but it does involve the command line.

• Go to the jhead site
• Scroll down to the Releases section and look for Pre-built OS-X Intel executable
• Right click on the jhead link on that row and choose Save Linked File to “Downloads”

At this point, I found that jhead was saved as jhead.txt. Oh well. We needed to do some command-line magic anyway.

• Open a terminal window and enter the following:

These lines do the following:

• Lines 1 and 2 navigate to the Downloads directory and remove the “.txt” from the name
• Line 3 makes the file executable by everyone on your Mac
• Line 4 places the file in a public area where everyone on your Mac can see it (you will be prompted for your password)

Stripping All EXIF Data

Sometimes I receive files that have corrupt EXIF data. I had a large quantity of scanned files in my collection that claimed to be scanned some time in 2038, while others insisted that they had been around since 1901. Neither situation is good, and I found that standard EXIF editing tools may fail to change these corrupt EXIF sections.

The answer is to blow away the EXIF data.

Download: Strip EXIF

This zip file contains a compiled AppleScript application. You can unzip it and place the application on your desktop. Safari will probably unzip it for you when you click the link.

To be safe, open AppleScript Editor and use it to open the Strip EXIF app to see its magic.

Now you can drop any number of JPEG files onto the Strip EXIF app and it will kindly eviscerate each JPEG, removing all traces of EXIF data.

Adding Basic EXIF Data to a vanilla JPEG

Some tools create JPEG files without EXIF date and time information within. This is typically the hallmark of photo manipulation software and dodgy scanner software. And if you happened to use the Strip EXIF app to rip out a bad EXIF block, then you will want to replace it with a proper data block so that you can still use camera date and timestamps.

Download: Add Basic EXIF

Again, unzip the file, place the app on your desktop, and then drop any number of JPEG files onto Add Basic EXIF.

The app will set the EXIF date to the file creation timestamp.

Spreading EXIF Timestamps

This is the real reason why I wrote these scripts. I couldn’t find a satisfactory tool on the market that would allow me to automatically spread out the shooting times for a series of images.

Why would anyone want to do this? Because some processes give you fifty JPEG files all with the exact same creation time and exact same shooting time. I like to use file renaming tools to incorporate the shooting time in the filename, so that files sort by chronological order. This doesn’t work if all of the timestamps are the same.

So I wrote a little app that adjusts the first photo by one minute, the second by two minutes, and so on. If there are fifty photos, then the last one will have its shooting time adjusted by fifty minutes.

The result is a series of photos/scans that have different timestamps.

Download: Spread EXIF Timestamps

Again, please look at the short program before you run it.

Closing Thoughts

I hope that my favorite tool implements these tricks soon (A Better Finder Attributes, I’m looking at you!), but until then, I will be dropping my files onto these three little droplets.

The jhead tool is so versatile that I will probably end up with a whole slew of similar droplets that will do all kinds of spiffy stuff. Nevertheless, I would rather the commercial products already provided these features. Not everyone likes dipping into AppleScript and the command line!

If you live in a slightly older home, such as mine, you occasionally might want to know which circuit breaker or fuse controls a particular outlet. Besides making it more convenient to disable the power for repairs, some of us have to deal with easily overloaded circuits that weren’t meant for all of the modern gadgetry we depend on.

Every homeowner can benefit from having a good map to their home outlets and circuit breakers, and a PDF scan of this map can make it extremely convenient to find two years later when you forgot you ever made it.

My Map

Last week my wife was asking if she could run her Jiffy Steamer in the bathroom, or if it would trip the breaker. I remembered making my cheat sheet, so I simply brought up Spotlight on my Mac and typed in “home circuit” and was rewarded with this document:

It’s nothing fancy, but it gets the job done. I quickly scrolled to the bathroom and identified the circuit that she was using. A quick scroll through the other rooms showed that she would be safe as long as she turned off the air conditioner in the bedroom.

Making a Circuit Map

There’s a little bit of process involved here, and it helps if you have someone else to help you.

Get a notepad and scrawl a rough sketch of each room in your house that has electrical outlets, switches, and lights. Don’t forget the basement, garage, and attic. Draw a rough sketch of each electrical outlet/switch on the maps. You can see in the image above that I simply drew a little box for each outlet and a box with bumps on it for a set of switches.

Then, shut off a single breaker and go around the house to see everything that lost power. Take a small desk lamp with you or, better yet, a proper line voltage test light, and test every single outlet until you identify the ones that are off.

Every time you find an outlet or switch that is off, write the breaker number on its spot on your map. You can see that breaker 19a and breaker 7 both control the living room in my house.

Now turn that breaker on and then turn off the next breaker and repeat the whole process.

It may take ten or fifteen minutes to make the first round-trip, but with each new breaker you have fewer things to test. You really only need to test outlets or switches that have not been identified yet.

When you are done, scan in everything and give the file a nice long descriptive name. Throw in some keywords if you are indexing your files in some application.

Using the Circuit Map

If you are concerned about the load on a given circuit, you can go through the whole document and look for every matching number (such as the “19a” from my living room) to see how many devices are on that circuit.

If you need to shut down power to a switch or outlet for any reason, find its breaker on your map, shut off the breaker, and then test the outlet with your line voltage tester before you do anything else. Even though you know the right breaker, you must always double-check that the circuit is dead before performing work.

By the way, my wife has had that Jiffy Steamer for years, and she absolutely loves it—it probably ranks right next to her iPad as all-time coolest and most useful products.

My situation is worse—both my wife and I are musicians, to one degree or another. Throw in the fact that she is a music teacher and you can imagine just how many pages of sheet music there are filling bins and flexing cheap shelving in my house.

What do I have and Where is it?

The biggest problem we face is knowing what we have and where it is. I have hundreds and hundreds of pages of classical and jazz guitar sheet music, but if I need to find Villalobos’ Choros no. 1, where do I look?

Shortly after I bought my ScanSnap, I began scanning in all of my sheet music (I have left much of my wife’s collection untouched—I’m sure you’ll understand). In most cases, I simply hacked the spine off of the original book and fed the sheets through the scanner. Now, I have less paper in the house and my music is searchable.

In most cases I didn’t bother to run OCR on the documents since there is little in the way of printed words on most sheet music that is worth indexing. I did take care to name the files well.

If you ever hope to find your music on your computer, make sure you include at least the composer/artist and song title in the file name.

Is this really cutting down on paper?

Whenever I find what I’m looking for I might play it directly off of the computer screen, but it is more likely that I’ll print it out. Doesn’t this kind of negate the idea of removing paper from my home? Not really. Think about it—most sheet music is never played. We have books with hundreds of songs in them and we play only  a handful. That’s just the way it is.

The fact that I print out five or ten pages in a month does not negate the many hundreds of pages that were scanned and then recycled.

Great for Music Lessons

I started taking jazz lessons again a month or two back, and my teacher gave me some lead sheets, with all kinds of useful annotations on them. As soon as I was home, I scanned those babies in, so I would not risk losing the valuable information. I also went through all of my notes from prior lessons and scanned them in as well. These kinds of things are precisely the sorts of paper that tend to get lost in some mismash of unsorted music.

Now, I can type in “Four” in my favorite PDF library application and find the lead sheet for Miles Davis’ Four.

Maybe you don’t have that many notebooks full of music lesson notes, but when you have been trying (poorly) to learn for as many years as I have, those notebooks begin to proliferate. Just scan them all in, give them some good filenames, add some keywords to help, and you’re in business.

What about copyright?

It seems that the jury is still out on digitizing works you own. There’s one fellow who made a right awesome device for scanning in textbooks in minutes, by photographing the pages. That guy’s machine has spurred much debate about whether or not you have the right to digitize your own stuff.

On the one hand, you bought the book and paid for it, so it would seem that fair use covers this; on the other hand, publishers are eager to monetize digital media, reselling the same works to you if they can.

So, is Daniel Reetz’s butt-kickin’ book scanner legal?

That would depend on who you talk to, says Pamela Samuelson, a professor at University of California at Berkeley, who specializes in digital-copyright law. Trade publishers are almost certain to cry copyright infringement, she says, though it may not necessarily be the case.

Google was recently forced to pay $125 million to settle with angry book publishers and authors who claimed copyright infringement as a result of the search giant’s book-scanning project.

But not so individual users who already own the book, says Samuelson. If you scan a book that you have already purchased, it is “fine, and fair use,” she says. “Personal-use copying should be deemed to be fair, unless there is a demonstrable showing of harm to the market for the copyright at work,” says Samuelson.

(Source: wired.com)

Here’s another take on this:

Question

I bought a book for school, can I make a copy of the book for my own use to write on so I don’t write in the book and can get my money back when I return the book to the campus store.

Accepted Answer

You have the right to make a copy of the book you purchased as long as you are using the copy for your personal use. The copyright laws merely prevent you from making copies to sell or distribute.

(Source: justanswer.com)

Of course, if you go passing your PDF documents around to all of your friends, all bets are off.

Final thoughts
Music is a hobby that seems to accumulate great stacks of paper, but these music sheets are peculiar in that you only need one or two out of every hundred. Why not digitize the whole lot and keep those book shelves from sagging?

I am of the opinion that rather than allowing an application to shuffle your life randomly, why not do it the old fashioned way and move your documents into folders of your choosing?

This article discusses some of the advantages of old-school folder management and gives a few hints along the way.

Why bother?

By creating your own well thought out folder structure, you gain the following advantages:

• You can find something fairly easily without needing to launch the special app.
• You can copy reasonable subsets of your document sets to friends or for backups.
• Someone else can find something without needing the special app.
• You can place files in a common network drive that others can see, from PC, Linux, or Mac.
• You do not lose all of the metadata about your files if the document management app ceases to exist.

People have been managing their documents this way for decades, so this is not anything new. What is new, however, is that folks don’t necessarily see what flexibility they give up when they allow the computer to squirrel things away on their behalf.

What kind of folders?

In short, pick some categories of documents that you will be filing, and optionally pick a timeframe which to partition the folders. This mirrors what we do with paper folders, doesn’t it? We create dozens of manila folders with tabs, and optionally create subsets of these files by date (e.g. Receipts, 2009).

One key difference helps us: Computer folders enjoy one feature that their physical counterparts lack—they can be nested several layers deep.

A few examples are probably in order…

I like to keep several kinds of scanned documents relating to day to day home paperwork. Over time, it has become clear that I scan lots of receipts, health insurance papers, banking papers, bills, and … everything else.

As such, I created the following top-level folders: Banking, Bills, Health Insurance, Receipts, and Miscellaneous.

Over time, they start to get stuffed to the gills with things, especially the Bills and the Receipts folders. My answer to this was to split them out by date. Within each category folder I have subfolders by date. This is because some categories need lots of years, while others might not need to be broken down by date at all.

Digital photos are a different creature: I feel that the date of the photo is the most important piece of information, and subject matter is secondary. For this reason, I store my photos in a series of top-level folders labeled with the years.

With photos I have a three-level system: Year, Month, and Subject. For example, within the Photos folder there is a 2009 folder. That contains a 2009-02 folder, and that one contains a folder called Cats. There are many ways to arrange these, I have chosen this approach.

I like iPhoto as much as anyone, and I use it for my photos. The difference is that, for me, iPhoto only holds a copy of each photo—the original photos are all stored on a NAS using the file structure I describe above.

Put a little thought into it and come up with a system that works for you.

Closing thoughts

We are looking for ease of use here, as well as avoiding lock-in to some proprietary app. We also want it to be easy to back up specific bits of the data and share specific bits.

By looking at my example above, you can see how easy it would be to find a bill from 2009. By following a specific naming convention, you can see that each document is fairly descriptive as well. You don’t need DEVONthink or its brethren to tell you how to find the Allstate bill from June of 2009. In addition, the folder names are now easily searchable by my operating system, as are the filenames.

This might create extra work for you in the beginning, but do you really want to be at the mercy of someone else’s application?

Oh, and about making those folders? There are applications out there that can generate a bunch of folders for you following your own chosen rules. One I use is The Big Mean Folder Machine.  I wouldn’t want to depend on an automatic system for daily use, but as a one-time jump start, tools like this can work wonders.

Don’t forget to back up your files!