I like the playlist tools that iTunes provides, and I find it very convenient to create Smart Playlists or to create Genius lists. However, I want to keep my Master Library of music elsewhere, far from iTunes. Wouldn’t it be great if I could share these playlists on my home network in a completely nondenominational format? Wouldn’t it be nice if my Squeezebox server would have those same playlists available? And wouldn’t it be spectacular if those playlists would magically appear on a network drive whenever the lists change in iTunes?

Read on for details on how I accomplished this and to download the free utility I wrote to handle this task.

The Problem To Be Solved

In order to understand what this application does, you need to know what I was trying to accomplish.

I have a NAS drive on my network that contains my Master Library of mp3 files. The files on this drive are located under /media/music.

I have a Squeezebox server, running on an Ubuntu machine. This server is used by Squeezebox devices to play music anywhere in my house. It holds a copy of the Master Library, located under /srv/squeezebox/music

What I want: I want my Mac to magically push proper m3u playlists to the Squeezebox server—the Squeezebox server understands m3u playlists.

What I don’t want: Squeezebox supports direct access to iTunes. But I don’t want to do this because I don’t want to have to run Squeezebox on my Mac. Why did I bother setting up a dedicated server if I have to keep my Mac running?

Why is this challenging: Even if I could point Squeezebox to the bare iTunes database file, all of the file paths are wrong. My master library might have a song at …/artist/album_name/disc 1/, while iTunes might put it at …/Artist/Album Name/, omitting the disc 1 folder and reformatting names.

My Solution

I wrote a small command line application in Perl that supports the following:

• Paths to songs in Apple’s mysterious black box are converted to nice paths to the same songs in my golden Master Library.
• Even if a song name changes or a folder path changes, the original master copy is found.
• All playlists are generated in m3u format.
• Smart playlists and Genius playlists are supported.
• The generated playlists are automatically moved to either a local directory or a remote server, using scp.
• Network shares are automatically mounted at beginning of process and unmounted at end of process.
• You can provide a file path to be prepended to all playlist entries.
• Exits early if the iTunes library has not changed since the last run.
• Supports exclusion patterns in order to skip some playlists.
• A local Squeezebox server can be automatically pinged to cause it to refresh playlists.

Here is the full documentation for the Playlist Mapper application.

Basic Installation

Please note that this is a command line program. Some day I might figure out how to make a pretty GUI installer for it, but for now you have to get your hands a little dirty in order to install and configure.

Download the application here: plmapper

This is a Perl script that uses many standard utilities already present in a Snow Leopard OS X installation. You should not need anything other than the plmapper file itself.

Unzip the file and place plmapper in your home directory.

Open a Terminal window and make the script executable:

chmod u+x plmapper

Read the man page for the application by doing this:

./plmapper --man

There are many command line arguments, but you don’t need to worry about most of them. Create a file called plmapper.config in your home directory and put any arguments you need in that file, as name-value pairs.

Example:

itunes=/Users/Bozo/Music/iTunes Music Library.xml
dest=/Users/Bozo/Desktop
library=/Volumes/media/music

Once you have added all of the configuration settings, go ahead and run it:

./plmapper

You should see plenty of status information go by as each playlist is processed.

Making it Run Automatically

There are better ways of doing this, but I’m a command-line commando, so I simply added the following cron entry (see crontab):

0 22 * * * /Users/Me/bin/plmapper >/Users/Me/plmapper.log 2>&1

This causes the plmapper program to run once daily at 10pm, logging output to a file called plmapper.log in my home directory.

Of course, this can be tweaked to taste; it doesn’t matter if you run it hourly since the app skips any heavy lifting once it sees that the iTunes library file hasn’t been touched.

How It Works

The application performs the following basic steps:

• Mounts network shares using mount_smbfs.
• All music files in the external non-iTunes Master Library are listed in an internal data structure, sorted by file size.
  This allows us to quickly find a “short list” of possible candidate files that match a given source file. This works because mp3 files have fairly random sizes.
• The iTunes database XML file is processed using an XSLT transformation to generate a set of basic m3u playlist files. The xsltproc command line utility is used to run the transformation.
• All paths in the playlists are converted from escaped URI syntax (e.g. %20 for space) to regular UTF-8 characters.
• Each path is resolved to a single music file (i.e. an mp3 file) in the Master Library.
  This is done by obtaining the size of the iTunes version of the file and then looking this up in the internal list created in the first step.
• Once a short list of candidates is found, the first few thousand bytes of each file are compared until a match is found.The risks of choosing the wrong file are low: all that will happen is that a song might be switched in the playlist accidentally. As such, it isn’t worth scanning the full file to ensure they are exact matches.This algorithm works quite well, and can detect music files that have been renamed and moved around.

• The playlist files are now converted from utf8mac to utf8 using the iconv command line utility.This process collapses wide UTF-8 characters, consisting of a letter plus an accent add-on, into their single-letter equivalents. This process is known as Unicode Normalization, and this tool uses Normalization form C: Compatibility decompisitiion followed by canonical composition.
• Playlist files are now copied to a local or remote directory, either as direct file copy or via the scp utility.
• Optionally, a special call is made to a specified Squeezebox server to tell the server to refresh all playlists.
• Any mounted shares that were mounted by this process are unmounted via diskutil unmount.

Limitations

• Nested folders of playlists currently act funny. The lists will come over, but you may get two lists. You can use the exclude option to prevent this.
• Songs with non-ASCII characters in the file name might not be recognized by all players. Squeezebox has a problem with many special characters—I tested this by adding songs like Águas de Março.mp3 using Squeezebox’s own playlist editor, and when I reloaded the list the songs were not there. This is a bug in Squeezebox server.
• Occasionally the file matching algorithm might pick the wrong song. In an effort to speed up processing, the program only looks at the first few thousand bytes of files when comparing. It isn’t a big deal if the wrong file is added to a playlist, is it?

Closing Thoughts

As often happens, this program was written because I had an itch that needed to be scratched. Now that I went through all of the hassles, I hope others can benefit.

Through good fortune, I am working in a Macintosh environment. This means that all of the important tools were already present on my machine: perl, xsltproc; mount; iconv; scp; nc; and others. If I were writing this for a PC, I probably would have written the whole thing in Java—a more familiar language to me, but one that comes with its own configuration issues.

I plan on updating plmapper over time, and if anyone has suggestions, please let me know.

And if a Perl guru wants to tell me my Perl skillz are subpar, go right ahead—but be gentle, and let me know how to improve the app.

[Update]

Fixed a couple of bugs related to path and “qx” calls, thanks to Chris. Updated version here.

I have a nice iMac that I use daily; my wife has a MacBook Pro which she has taken quite a liking to. And I carry around my iPhone (she really couldn’t care less about smart phones). It looks like these devices all converge on and overlap the territory of the iPad. Again, what’s the point?

I walked into the Apple store last Monday expecting to enjoy a few minutes of playing around with an over-sized iPod Touch, and then walk out. Then it hit me: they did it again—they created a device, akin to the iPhone, that is so slick and easy to use that you must handle one and play with its features before you can truly understand.

Alex Payne put it quite nicely in his blog:

“Human-computer interaction has found a sweet spot on the iPad. It’s all the power of desktop computing, plus the valuable constraints of mobile devices, minus the limitations of both. It just makes sense. Use one for a couple hours and your desktop or laptop will seem clumsy, arbitrary, and bewildering. It is, simply, how (most) computing should be.

You can be as cooly aloof as you like about the device, but it won’t change the fact that it’s a fundamental step forward in computing. … [I]f you work in tech, you should spend some time with an iPad. If it doesn’t change the way you think about what you do, you’re either a genius or an idiot.”

— The Moderate’s Position on iPad Openness

He’s right, you know.

As soon as I held the device in my hands it was clear why this device has earned its own new niche that it was wedged into, between smart phones and laptops. The touch interaction that was so revolutionary with the iPhone has become more palpable, and more natural. I find myself gently sweeping my hand across the screen as I read the newspaper, watching the words gently glide by.

It really sank in when I looked at how my wife uses her MacBook Pro. She lays in bed with the machine in her lap, listening with headphones, as she goes through her email, listens to iTunes, searches out videos of old friends in Brazil posted on YouTube, and does some Google searches for whatever is on her mind. Meanwhile, the machine’s legendary thigh-roasting fans are running and she fidgets and fumbles with its bulk.

Everything she does with her MacBook is better with the iPad. It is more like a TV than a computer in the sense that you simply turn it on and choose what you want to do, with no knowledge of its internals. There are no fans blasting searing heat. The device is not cumbersome; she can curl up with it like a good book.

And it does Netflix.

What more could you want?

Well, there are few things I see right away that I would like, but for the most part I want more proper iPad apps. Old iPhone apps offer two equally unpleasant views: either you use the app in a horribly cropped iPhone-sized letterbox view, or the app is displayed in grotty pixelated full screen mode. Fortunately, folks are coming out with new iPad apps every day, some are even free upgrades if you own an app on the iPhone.

The first problem my wife will encounter (when I finally give her my iPad, as promised) is that she will want to print something from it. Printing doesn’t seem to be in the iPad’s repertoire. I have to admit that printing is a bit of a heavyweight for such a handy dandy device. She will still look at me and say “But it should be able to print.” And she’ll be right.

The second real issue I have with it is file management. There just is no simple way to move files onto an iPad: all file management is kludgy at best, usually involving iTunes. This was not so bad with the iPhone because our expectations are lower; after all, it is a cellphone first and foremost.

But the iPad is different. Its name screams “Documents” and begs us to flip pages with our bare hands. Why do all document transfers have to involve web browsers, email, and iTunes?

There are many different ways file management could be handled, but the way iPhones and iPads deal with documents and files is so un-Apple. This little bit of tarnish distracts from the beautiful polish of the device.

But I love it anyway

I can’t really criticize much more about my iPad. It does exactly what someone like my wife needs with little hassle, like a handheld flat-screen TV with cool features.

In other words, the iPad excels at being a computer for entertainment.

Check out some of the (currently) free newspaper apps. If you like Popular Science, drop a five-spot on their interactive magazine. You’ll like it.

And did I mention that it does Netflix?

My situation is worse—both my wife and I are musicians, to one degree or another. Throw in the fact that she is a music teacher and you can imagine just how many pages of sheet music there are filling bins and flexing cheap shelving in my house.

What do I have and Where is it?

The biggest problem we face is knowing what we have and where it is. I have hundreds and hundreds of pages of classical and jazz guitar sheet music, but if I need to find Villalobos’ Choros no. 1, where do I look?

Shortly after I bought my ScanSnap, I began scanning in all of my sheet music (I have left much of my wife’s collection untouched—I’m sure you’ll understand). In most cases, I simply hacked the spine off of the original book and fed the sheets through the scanner. Now, I have less paper in the house and my music is searchable.

In most cases I didn’t bother to run OCR on the documents since there is little in the way of printed words on most sheet music that is worth indexing. I did take care to name the files well.

If you ever hope to find your music on your computer, make sure you include at least the composer/artist and song title in the file name.

Is this really cutting down on paper?

Whenever I find what I’m looking for I might play it directly off of the computer screen, but it is more likely that I’ll print it out. Doesn’t this kind of negate the idea of removing paper from my home? Not really. Think about it—most sheet music is never played. We have books with hundreds of songs in them and we play only  a handful. That’s just the way it is.

The fact that I print out five or ten pages in a month does not negate the many hundreds of pages that were scanned and then recycled.

Great for Music Lessons

I started taking jazz lessons again a month or two back, and my teacher gave me some lead sheets, with all kinds of useful annotations on them. As soon as I was home, I scanned those babies in, so I would not risk losing the valuable information. I also went through all of my notes from prior lessons and scanned them in as well. These kinds of things are precisely the sorts of paper that tend to get lost in some mismash of unsorted music.

Now, I can type in “Four” in my favorite PDF library application and find the lead sheet for Miles Davis’ Four.

Maybe you don’t have that many notebooks full of music lesson notes, but when you have been trying (poorly) to learn for as many years as I have, those notebooks begin to proliferate. Just scan them all in, give them some good filenames, add some keywords to help, and you’re in business.

What about copyright?

It seems that the jury is still out on digitizing works you own. There’s one fellow who made a right awesome device for scanning in textbooks in minutes, by photographing the pages. That guy’s machine has spurred much debate about whether or not you have the right to digitize your own stuff.

On the one hand, you bought the book and paid for it, so it would seem that fair use covers this; on the other hand, publishers are eager to monetize digital media, reselling the same works to you if they can.

So, is Daniel Reetz’s butt-kickin’ book scanner legal?

That would depend on who you talk to, says Pamela Samuelson, a professor at University of California at Berkeley, who specializes in digital-copyright law. Trade publishers are almost certain to cry copyright infringement, she says, though it may not necessarily be the case.

Google was recently forced to pay $125 million to settle with angry book publishers and authors who claimed copyright infringement as a result of the search giant’s book-scanning project.

But not so individual users who already own the book, says Samuelson. If you scan a book that you have already purchased, it is “fine, and fair use,” she says. “Personal-use copying should be deemed to be fair, unless there is a demonstrable showing of harm to the market for the copyright at work,” says Samuelson.

(Source: wired.com)

Here’s another take on this:

Question

I bought a book for school, can I make a copy of the book for my own use to write on so I don’t write in the book and can get my money back when I return the book to the campus store.

Accepted Answer

You have the right to make a copy of the book you purchased as long as you are using the copy for your personal use. The copyright laws merely prevent you from making copies to sell or distribute.

(Source: justanswer.com)

Of course, if you go passing your PDF documents around to all of your friends, all bets are off.

Final thoughts
Music is a hobby that seems to accumulate great stacks of paper, but these music sheets are peculiar in that you only need one or two out of every hundred. Why not digitize the whole lot and keep those book shelves from sagging?