I’m serious. You don’t need to know anything at all about Linux to use one of these. Just download the image, install, and you suddenly have a full featured NAS file server, or you might have a database or a source code repository.

Last year I wrote an article on how to set up a NAS device using Ubuntu Linux. I have been a fan of Ubuntu since the start because it is a very easy distribution to install and configure. The down-side of using Linux has always been the fairly steep learning curve. Before you can get around to using the server, you need to get down in the weeds with configuration files and other stuff.

TurnKey Linux changes all of that.

Painless Installation

A few weeks back, I was setting up an aging PC as a standalone wiki server for a small office—this machine was going to provide a place for the office staff to document their procedures, how-tos, and other things.

I was about to set up an Ubuntu server, as I have done before many times, and install MoinMoin, like I did some months back. I remembered that it was a bit of a pain to get everything tweaked just right, so I did a quick check to see what kind of standalone wiki options were available online.

This is how I found TurnKey Linux. This project is all about single-purpose preconfigured Ubuntu server images.

One of those preconfigured images happens to be a MediaWiki appliance—the wiki engine behind Wikipedia—and I was in business.

The installation took about fifteen minutes, with very little user interaction. I answered a few basic questions and the installer took over from there. As soon as the install was done, the machine rebooted and displayed a message on the monitor with the IP addresses where you can browse to from any other machine.

Full Featured

The work that has gone in to these appliances is amazing. In fifteen minutes I had installed a complex configuration that has the Apache, PHP, MySQL, MediaWiki core, as well as maintenance utilities such as a neat tool that provides a Flash-based pure-AJAX-based SSH command line in a remote browser (i.e. your browser becomes a terminal). Even someone with Linux experience would have to spend quite a bit of time fiddling around with different packages and configuration options in other to provide the same functionality that TurnKey gives you out of the box.

As with most open source projects, the documentation is about 80% complete, with deep detail in some areas, but leaving others fairly sparsely documented. But don’t let this deter you: in most cases users know how to use the product they are installing (e.g. MediaWiki) but don’t want the hassle of configuring it on Linux. That’s where TurnKey shines.

Some Examples

In minutes, you can set up a NAS device. If you want to try advanced content management in your office, try Joomla or Drupal.

If you are working on a small project team and want to protect your source code, try Redmine or Trac and do your bug tracking using Bugzilla.

And while you are at it, you can document your organization’s working practices using a wiki such as MoinMoin or MediaWiki.

Don’t forget to back it up!

As with any computer, you should include your new TurnKey appliance in your backup strategy. The nice thing is that you don’t really need to care at all about backing up Linux or the other software; just back up the data. I don’t need to back up my entire MediaWiki machine; I just need to back up the database and image files. If anything goes wrong, you can rebuild the TurnKey appliance from scratch in minutes and then restore your data.

To save yourself some pain, keep notes on any small tweaks you made to the configuration.

One Machine, One Purpose

These disk images share common Ubuntu underpinnings, but they are referred to as Appliances because they turn your PC into a purpose-built appliance.

This means that if you want a content management system and you also want a ticket management system, you will need two old computers—not a rare commodity these days.

Take a look at what they have to offer and give TurnKey a shot—specialized software used in corporate environments is now within reach of small offices at the right price.

Among other things he discusses options for prioritizing and cutting down on the total volume of stuff you plan on keeping, digital or otherwise.

“Backup, backup, backup” made number four on his list.

And finally, he provides a few notes on some helpful free organizing software. I think I’m going to check out that Know Your Stuff application he mentioned.

Five Tips to Paperless Finances (moneytalksnews.com)

Imagine that you were to become temporarily incapacitated for whatever reason…

• Can a family member log in to your computer, as yourself, in order to access your files?
• Can your spouse access your online banking details so the bills can be paid?
• Can your family find your insurance information that you scanned and filed away?
• Is there someone who can log in to any online accounts that need care and feeding?

Not a pleasant subject, indeed, but one that worries me from time to time.

One way to address these needs is to keep all of your passwords and so forth in one special place, using a password safe application, and make sure someone else has the access code. For example, you can use a tool such as 1Password or SplashId to store hundreds of secret bits that you use all the time, and your family might need.

You might consider writing down the master passwords that control your life and sealing them in an envelope that you provide to a trusted family member. Since this is such a great security risk if found by the enemy, you might want to omit any identifying information from the note. Impress upon them the need to secure the document very well.

Perhaps you can choose the same master password with your spouse, with one relatively short password locking your computer and a long secure password locking your password safe application.

Regardless of how you address these issues, sit down with your better half (or trusted family member) and review where documents are and how to access them.

Apparently this is not the case

It seems that the TSA has leaked their official document of airport security guidelines. ABC News says Online Posting Reveals a “How To” for Terrorists to Get Through Airport Security

A Rookie Mistake

Look at the screenshot of the document at the top of this post. Even though a certain part of the document has been blacked out, it is possible to select the text and copy/paste to find out what is hidden behind the black text.

What kinds of things are listed in this document?

• Photographs of all kinds of official ID cards. Ever wondered what a U.S. Senator’s ID card looks like?
• Procedures for calibrating equipment, such as where guns should be hidden for the testing and such.
• Guidelines for who gets searched and who doesn’t.
• Guidelines for what objects get searched and which don’t.
• And much much more!

In other words, this was a most unfortunate event.

See for yourself—ABC News (and others) have posted the document with redactions removed.

Easy as Pie

Here’s a screenshot of the original document, opened in Adobe Acrobat Professional.

As you can see, it was a trivial matter to use the TouchUp Object tool to gently slide the black rectangle off of the secret stuff (I have blurred the text here, though you can read it from ABC News if you wish).

If you are working with confidential documents that could potentially cause disaster if leaked, please learn how to redact your documents correctly!

I was reading a post on a board I frequent where a person was describing exactly this kind of activity—removing sensitive information from PDF documents. Several suggestions were made, but one individual suggested opening the file in Acrobat Pro and replacing the sensitive text with good old Lorem Ipsum.

It was at that moment that I recalled a peculiar feature of the PDF file format: it is designed to support nondestructive updates, allowing people to make vast changes to a PDF document while still retaining the original document, fully intact. I did a few experiments and was surprised with the results.

A Brief Note on the PDF File Format

For the geeky types among us, one place to begin is this article:

Portable Document Format: An Introduction for Programmers

The key points to get out of the article is this: A PDF document is comprised of several distinct sections, a Header, a Body, an “xref” Table, and a Trailer. At the very end of the file you will find the character sequence %%EOF

The PDF standard was designed to allow multiple updates to a document, while retaining the original version. This is accomplished by appending anything new to the end of the document, after the original EOF tag. The document will now have two EOF tags: one indicating where the original document ended, and a new EOF tag indicating where the new changes end.

If we wish to revert PDF changes, it should be a simple matter of opening the PDF file in a binary editor, searching for the first EOF tag, and deleting everything following.

A Simple Experiment

Let’s start with a proper secret document containing missile plans…

Suppose we want to obscure some special information in paragraph 37. We can open the file in Acrobat Professional and use its text editing features to swap in the venerable Lorem Ipsum text.

Here’s what it looks like after the switch:

You can see here that the first seven lines of text starting on paragraph 37 have been replaced with appropriate unreadable text.

Now, open the new PDF file in a binary editor (since PDF files contain a mix of text and binary, the editor must be a binary editor).

Note the %%EOF character sequence embedded in the text. This is the first EOF tag, indicating where the original file ended. All we need to do is place the cursor to the right of the EOF and delete everything to the end of the file.

Once we have done so, it’s like magic:

The edits that replaced lines of paragraph 37 with gibberish have neatly been undone!

More Details

From the PDF Intro document linked earlier:

“The trailer, it turns out, plays an important role in the way PDF implements incremental updating. The key concept to understand here is that a PDF file is never overwritten, only added to. That goes for all portions of the PDF file – even the trailer itself, and the end-of-file marker. In other words, a multiply-updated PDF document may contain multiple trailers – and multiple end-of-file markers! (There may be numerous occurrences of %%EOF.) Each time the file is edited, an addendum is written to the tail of the file, consisting of the content objects that have changed, a new xref section, and a new trailer containing all the information that was in the previous trailer, as well as a /Prev key specifying the byte offset (from the beginning of the file) of the previous xref section. The cross-reference info will then be distributed across more than one xref section. To access all of the cross-references, the reader must walk the list of /Prev keys in all the trailers, in reverse order.

Space doesn’t permit a detailed exploration of updates here, but you can find several examples in Appendix A of the PDF 1.3 specification (available at http://partners.adobe.com/asn/developer).”

Summary

It is important to understand that the PDF standard allows for appended updates to files that leave the original document intact, regardless of how drastic the changes are. If you are intent on redacting text from PDF documents, do not depend on simply deleting the secrets using a PDF editor—you must use a proper redaction tool that addresses these issues correctly.

That said, I did some experimenting with a few utilities (Apple Preview, PDFpen, and Adobe Acrobat Pro) and found that some write the file from scratch each time, with no lingering cruft from former versions, while others respect the original intent of the PDF standard. This means that you can’t trust that older revisions are being retained in your file and you can’t trust that they aren’t.

Be conservative: use a redaction tool for secrecy and proper backups for versioning.

As it would turn out, there were several thousand other documents on a file server that shared the same funny behavior. By the time we were done struggling with this problem I had gained new respect for PDF corruption issues and their prevention.

The Problem

We were looking to load a few thousand of these scientific reports into a fancy-schmancy new database, with linguistics searching and other bells and whistles. Much to our chagrin, these documents just weren’t loading, and we couldn’t understand why. They were text documents, with some embedded images, but mostly straightforward text.

Here is an excerpt:

And you can tell that it is right and proper text because when I blow it up all the way, the fonts are nice and smooth—this isn’t just an image of text.

But if I copy and paste that particular paragraph into any handy editor (Notepad, in this case), this is what I see:

And as far as I know, at this point the actual text is beyond the reach of average folks like me. We tried, believe me we tried.

What went wrong?

A quick Google of the subject led us to understand that many PDF generation tools embed subsets of fonts, with nonstandard mappings from the text to the font.

This fellow explains it nicely:

“The PDF file does not contain all the information to extract the text. The problem is that a character in a PDF file may not contain information what “real” character it relates to. Some PDF generators do a pretty bad job when they embed fonts into PDF files. They use a proprietary encoding mechanism (e.g. 1 is A, 2 is B, 3 is C, …) in both the embedded font and when they place glyphs on the page. Without a table that implements the reverse (e.g. character code 1 is ‘A’) you cannot extract text from such a file.

There is nothing you can do (besides to complain to whoever created the PDF file, and the author of the software that created this file).”
— from khkremer on experts-exchange.com

As it would turn out, many of the reports had been generated by printing to Adobe Distiller from Microsoft Word. It would seem that the default settings used for Distiller included the “totally hose my document content” switch.

The Solution

We fretted over this quite a bit. These are important scientific reports, and there is no way to easily ungarble them. We finally ended up contacting the Abbyy Finereader folks and trying out their OCR toolkit for Linux: not only did this product make fast work of running optical character recognition on the sample document, but once we had a script running, we managed to blow through the 10,000 pages the trial license gave us, in a day or two.

Imperfect, at best

I am happy that we were able to salvage the bulk of the electronic knowledge found within those thousands of files, but our work barely scratched the surface.

For example, most of these documents have rich bookmarking of sections and keywording, such as this (content tastefully blurred on purpose).

In addition, scientific documents typically have loads of tables full of numbers. Though it is possible to mine this data with a good OCR tool (the FineReader API provides tools for just this purpose), the tables are far more difficult to extract correctly once the original text information is lost.

Final thoughts

I wrote a few weeks about document formats, mentioning the PDF/A document standard. This is worth investigating, regardless of what your document needs are.

If our thousands of files had been originally generated as PDF/A, it is certain that we would have been able to copy/paste from them without problem: PDF/A prohibits such font shenanigans as were perpetrated on our garbled reports.

In the end, our OCR sledgehammer approach worked like a charm, and is probably sufficient for our needs. Text mining is a pretty slushy business, so no-one will complain if there are a few typos on each page—if they find the doc in a search, they can print it and read it the old fashioned way.

I trust my digital life to PDF. Almost everything I scan and most documents I write eventually end up squirreled away somewhere as PDF documents.

Have you ever considered just how portable those documents really are?

What’s wrong with PDF?

It seems strange to question the portability of these files, doesn’t it?

For the past ten or fifteen years Adobe has been providing Acrobat Reader and singing the wonders of their new universal document format. And it seemed to be all that, too—regardless of the audible groan we give when Acrobat launches after we click a link, isn’t it amazing that we can download press-ready copies of our income tax forms, that are guaranteed to look exactly the same when you print them as when I print them? Read on to see what dangers lurk within.

What’s the Problem?

In order to understand the nature of the PDF portability issues, one need only look as far as the web browser for an analogy. Consider how the web browser went from a barebones tool that could display a simple language, HTML, in a neutral way, fitting the web content onto each user’s screen, to a memory hogging behemoth that is an integral part of your operating system. It didn’t happen all at one; it has been death by a thousand cuts.

Mirroring the evolution of web browsers, the PDF document standard has adapted over the years to include many bells and whistles such as embedded audio, video, and JavaScript. It is these features that chip away at the core purpose and raison d’être of the PDF standard.

An example: Font Issues

A simple example of the weakness of these extended PDF features is the humble text font. When your application generates a PDF document, there is the option of using 14 standard PDF fonts, local machine fonts, or embedded TTF or Postscript fonts.

There are 14 standard fonts that should be available by default in each PDF reader. These fonts are Courier, Courier Bold, Courier Italic (Oblique), Courier Bold and Italic, Helvetica, Helvetica Bold, Helvetica Italic (Oblique), Helvetica Bold and Italic, Times Roman, Times Roman Bold, Times Roman Italic, Times Roman Bold and Italic, Symbol and ZapfDingBats® (source)

Guess what happens when you set your document in Mona Lisa Solid ITC and then print to PDF and send to all of your colleagues? Does your friend’s machine have a copy of this font? Maybe, and maybe not.

As I was writing this, I planned on putting together a cute demo by saving a document set in Mona Lisa Solid ITC in PDF from my Mac and then opening it on a PC. Much to my surprise (and delight), I found that the default “Print to PDF” functionality on my Mac does, in fact, embed the font within the document.

Regardless, if you have always just trusted that the fonts would be identical across platforms, you could get quite a surprise when your friend tries to print your beautiful document.

PDF/A Standard

Some time back, Adobe recognized the need for a more tightly controlled standard, for creating really portable documents, instead of mere portable documents. This standard, dating from 2005, is referred to as PDF/A, where the A stands for Archive.

A key element to … reproducibility is the requirement for PDF/A documents to be 100 % self-contained. All of the information necessary for displaying the document in the same manner every time is embedded in the file. This includes, but is not limited to, all content (text, raster images and vector graphics), fonts, and color information. A PDF/A document is not permitted to be reliant on information from external sources (e.g. font programs and hyperlinks). (Wikipedia)

Basically PDF/A forbids all of the flashy stuff and sticks to the basics: good solid document rendering.

Banned features include:

• Audio and Video
• JavaScript
• Encryption
• Nonstandard metadata
• Transparent images

In addition to the loss of several features, PDF/A documents can be somewhat larger, due to the embedded fonts, and they might have rendering issues with images that depend on transparency.

With all that, it still sounds like an enticing concept. Many PDF tools speak fluent PDF/A. Check out your own toolkit and see if you can future-proof your documents a little more

Here’s more on PDF/A documents

Long-term digital archiving with PDF/A (The PDF Blog)
PDF/A (Wikipedia)
PDF/A – A new Standard for Long-Term Archiving (PDF/A Competence Center)

Now prudence would lead one to take careful steps and wait until all of the wrinkles are ironed out before starting. I was almost not prudent enough this week.

Mac OS X Snow Leopard

So folks have been talking about the new Snow Leopard operating system for Mac. Over the past year, Apple has been positioning this version as more of a “under the hood” upgrade that tightens things up rather than a glitzy overhaul of the user interface. No matter what they said it was, I figured that it was newer, and therefore better, than the current OS—Leopard–and I had to have it.

I ordered my copy last week on Amazon and sat down with a smile as I awaited its arrival. And then I thought about doing a few quick Googles to see how other people have been making out with Snow Leopard. I immediately happened upon a few upgrade guides like this one, providing sage advice about the upgrade process. They recommended the “slash and burn” method, starting from a clean hard drive, and I felt that was a good idea. Nothing better than a wipe and fresh install to make your machine zip along twice as fast. And therein lies a tale.

The first sign of trouble

As I was reading up on the Snow Leopard upgrade process, I happened upon lists of “unsupported software” and casually glanced at the lists, expecting esoteric tools only used by three über geeks in the audio recording industry or perhaps some exotic ray-tracing software. Much to my surprise, I saw two of my favorite applications, in a very very short list of troublesome apps: Parallels and EyeTV.

I immediately checked the versions and breathed a sigh of relief when I saw that my EyeTV version was safe. But, Parallels was another story… They have no plans for patching Parallels 3 to work with Snow Leopard, and why should they, when they can sell us Parallels 4!

So, I ordered my fresh copy of Parallels 4, from Amazon with a twenty dollar rebate. When it arrived, I spent an evening upgrading Parallels, and I thought I was all set for Snow Leopard.

Preventative Measures

Following the advice of the upgrade websites, and prior experience, I used Carbon Copy Cloner to make a full backup of my hard drive on a spare external drive. On a hunch, I turned on the drive that I use for Time Machine and had it do one final “Time Machine” sweep through the system before bidding adiu to Leopard.

I knew that I had all of my installation media for stuff like iLife and Photoshop Elements, and I had all of my license keys in electronic form. It would be a simple matter of mounting the backup drive, copying over my loads of documents, and peering into them to find keys.

The first attempt

I boldly inserted the Snow Leopard disk and booted from the DVD drive, selecting the “Slash and Burn” method of installation. I reformatted the hard drive and went off for dinner while Snow Leopard installed.

Trouble

When I got home that evening, I started the lengthy process of installing stuff. I suddenly realized that it was not as easy as I had hoped: it’s one thing to reinstall something like Microsoft Office, but there seemed to be more loose ends than I had considered:

• How would I migrate my Mail settings from the old image to the new?
• What was the best way to migrate the Address Book contents?
• iTunes is great, but it has tendrils in everything. Can I simply copy my old library to the new without messing up my iPhone, Address Book, or other linked stuff?
• How about those nice password tools such as 1Password and SplashID that keep your passwords safe and sound? I had no clue how to get their contents from the backup. I wasn’t sure if it was even possible to do so—perhaps I was supposed to have exported the data beforehand.

It was becoming clearer to me that I had not done my homework at all.

More trouble

My initial shock at the depth of the upgrade process led me to start making a list of applications and looking at what I needed for each one. I soon found out that Snow Leopard support is somewhat spotty in many applications. In particular, the FineReader for ScanSnap software that I depend on so much for my scanning work flow is not fully supported. Fujitsu says that they will have an update soon and to keep checking their web site.

My password tool, 1Password, is another problem child. It works only on 32-bit Safari, and Snow Leopard now runs Safari in 64-bit mode. Of course, a new version is coming, and I will probably have to pay for it, but it is still in beta.

There was quite a bit of chatter on the Web about whether Adobe Photoshop Elements would work on Snow Leopard, and the responses seem split fifty-fifty for now.

Three very important tools were in danger of running in limited mode or not running at all, so I had to throw in the towel.

Time Machine saves the day!

As I sat, humbled, before my vanilla install of Snow Leopard, I admitted defeat. I slipped the Snow Leopard DVD back in the drive and rebooted from the DVD. This time, I selected the “Restore from Time Machine” option and turned on my Time Machine drive.

Guess what? It worked perfectly! Unlike many software products, Time Machine does exactly what it promises.

Within a few hours, my machine was fully restored to the way it looked seconds before I made my first attempt at Snow Leopard.

A Final Word

Learn from my mistakes, and my salvation by the full backup. As much as you can’t wait to upgrade, please do the following:

• Inventory all of your applications that you really need.
• Obtain the installation media (download or CD) for every single one.
• Obtain the keys for every single one.
• Investigate whether you need to export data from any of them, and make a checklist for these exports prior to upgrade.
• Check the “Unsupported Software” lists that are out there for any red flags.
• Check the web sites of your most important apps for their official word.
• And finally, do a complete backup!

It’s amazing how many applications and weird little utilities we forget we have. How could I have possibly remembered that I compiled a custom copy of the “rsync” executable for my backup workflow? I would have lost that and had to figure out how to rebuild it on Snow Leopard.

And I haven’t even talked about making sure your documents make it safely onto the new machine. That’s a whole ‘nother story.

In case I forgot to say it, please make a full backup.

[Update: I'm giving Snow Leopard a rest for a few months]

It has been said that Time Machine allows you to do a full restore from bare metal, and I’m living proof: I have done exactly that twice in the past week, with astounding success.

Encouraged by an episode of the Mac Geek Gab where they talked about their experiences upgrading their existing systems to Snow Leopard, I decided I would give the upgrade-in-place option a try. I expected some things to not work well and others to be quirky, but here’s what happened…

The actual install was painless, taking an hour or so to complete. I then began to kick the tires to see what was broken.

It was clear where those 64 bits went: apps like Safari were positively zippy, and I was pleasantly surprised with each new application I launched. All of my special settings seemed to make it through alive, including my password manager, though I did have to re-enter some of my registration keys. All of my mail and contacts made it through well. I was able to sync my iPhone without incident.

I found a few apps that weren’t working correctly and I looked for newer 10.6-compatible versions. I found newer versions of Yep and xGestures.

I did note that there is currently no ad blocker available for Safari that runs in 64-bit mode. This is disappointing because even though I understand that Apple wants us to see their ads, I can’t imagine that they really want us to suffer from the flickering jumping dreck that should have ended with the hated “punch the monkey” banners of years gone by. The fact of the matter is, if I want that 64-bit speed and snap, I guess I have to watch ads.

The Showstopper

I decided to scan a document to see just how difficult it would be to get my workflow going again. Michael F, below, wrote the truth about the situation: the scanner works fine in certain modes, but the OCR software doesn’t.

He pointed out that it was a problem of the FineReader software looking for a specific bit of metadata in the PDF identifying it as a ScanSnap PDF. Sadly, that metadata string changed.

The Finereader software is looking for “Mac OS X 10.5.8 Quartz PDFContext”, but under Snow Leopard, the string is set to “Mac OS X 10.6 Quartz PDFContext” instead.

There are ways to tweak PDF metadata, and one of them is by using pdftk.

I went to the pdftk site, all ready to download it and start OCRing my PDFs. I was greeted with less than optimal news: they have a version compiled for Panther, a version of OS X from several years ago.

I knew it wouldn’t work, but I gave it a try anyway: the app told me it needed Rosetta to run. I could have installed Rosetta at that point, but I figured I wanted a proper compiled version.

From there, I looked into compiling the app on OS X 10.6. I should have remembered my struggles with this several months ago on a Solaris Unix box when I found that pdftk depends on a monster called GCJ that required about forty other software packages to compile—it seemed a gargantuan task that I wasn’t ready to begin.

On a hunch, I inspected the content of a new pdf and an old pdf, the latter still acceptable to FineReader. Though much of the file was raw binary, the metadata was in text at the end. A short sed script was all it took to swap the nice text string for the offending 10.6 one.

In spite of my best efforts, FineReader still rejected my hand-tooled PDF file. It knew that it was a bogus file.

I have looked into Abbyy FineReader several times before, as well as Fujitsu’s ScanSnap support, and was unimpressed. For two vendors that produce products that are at the top of their class—FineReader is arguably the best OCR you can get for Mac, and ScanSnap is the best document scanner for the common man—they sure do have miserable customer support.

It is as if neither company cares a whit about the Macintosh platform or their customers. While most other vendors are busily patching their products and giving hourly updates on their Snow Leopard compatibility progress, Abbyy and Fujitsu just don’t seem to care that their best-of-breed combo suddenly doesn’t work on Mac.

Once they get this sorted out (hopefully in the next few months) I’ll give Snow Leopard another try. In the meantime, I’m sticking with good old Leopard.

My own choice for document format is almost always Portable Document Format (PDF), but rather than just state this, I would like to consider some of the factors involved.

This is the first of a series of articles covering document formats. This article focuses specifically on the distinction between works in progress and finished product.

Two Kinds of Documents

In general, we can consider two broad categories of documents: working documents (works in progress) and archived documents. You can call these by many different names, but the fundamental distinction is still there.

Working Documents

These are documents that you are still writing. They share some characteristics:

• They must be retained in their original format, such as Microsoft Word.
• The formats are often very specialized. Quite often another tool can import such a document, but you usually lose something in the translation.
• You and your colleagues need to have the same editor software to view and modify the documents.
• They are often short-lived. This phase of a document’s life usually doesn’t more than a few months (though a template document might be kept for many years).
• A good backup strategy will need a short window between backups; these documents change often, so they should be backed up frequently.
• You may want to consider a document versioning strategy, so you can see how the document appeared at different stages during its life.

Here are some examples:

• Microsoft Word documents
• Visio diagrams
• Photos that you are still retouching
• Audio files that you are in the process of curating (e.g. applying ID3 tags)

Archived Documents

These are documents that are read-only, meant to be viewed but never modified.

• They often must be rendered in very precise ways, so each viewer sees the document as intended (consider a 1040 form from the IRS)
• They may be around for a long time.
• These documents should be less tightly bound to a particular software product. PDF vs. MS Word; JPG vs. Adobe Photoshop.
• They typically have a wider audience. You may share a work-in-progress with a co-worker or two, but a finished read-only document might be read by hundreds or thousands.
• Any user should be able to read these documents, with little effort.
• Your backup strategy is probably going to be more focused on longevity and less focused on frequency. These documents are in it for the long haul.

Why not start with a simple example?

Here is a screenshot of an application I use in my day job:

Just in case you did not recognize the unmistakable visage of this small molecule, I have labeled it appropriately.

This is an application called ChemDraw from Cambridgesoft, and unless you are a chemist you have probably never heard of it. My molecule is saved as caffeine.cdx in a format that only ChemDraw knows intimately (though there are other similar chemistry tools that can import this file format).

My point is simple: if your friend sent you a copy of caffeine.cdx, how exactly would you open it?

In contrast, here is a more accessible rendition of the same molecule in PDF format. Try it out; you should be able to view the molecule, and zoom in on details.

What if you had to show someone this document five years down the road? Do you want to have to chase down a possibly obsolete version of a very expensive application that might not even run on your operating system?

Obsolescence

Some time back I was sifting through some files on an old server at work that apparently had been written by me. Fifteen years ago I was attending night classes and writing many of my English assignments on a VAX running VMS at work (over my lunch break!). I was using some anemic version of WordPerfect that had been ported to VMS. This arrangement saw me safely through college, but was not conducive to long term document storage.

Do you have any idea what VMS directory structures look like? Maybe, and maybe not. Are these files compatible with the contemporary DOS versions of WordPerfect? Maybe.

Could I open these files on a Windows Vista machine in 2009 using Microsoft Word? With luck. What about using Pages from Apple iWork on my Mac running OS X? Doubtful.

Not only do we need to be concerned with special applications that only a select few (with expensive licenses) have, but we also need to consider that the file format might be obsolete beyond hope.

For an exaggerated example, consider the image of punched paper tape at the top of this article. I would have no clue what to do if I were given a roll of this tape.

Which do you keep?

Look at the characteristics of the document types listed above and see which one fits your document best. Quite often you will find yourself keeping both the original document and a PDF rendition. Indeed, this is what many professional document databases do.

If you can’t easily choose one, keep both. In most cases, I have found that I only need the PDF rendition for the long term and I couldn’t care less about the source document.

Summary

In the world of the paperless home, much of what we do is store digital copies of old documents for searching and possible reprinting some time in the future. Don’t make the mistake of keeping all of your documents only in their original editable format; you might just find yourself with a digital file that cannot be viewed!

Their network security department very kindly (and politely) informed me that they had received a “cease and desist” order from a particular game publisher. They had included the game publisher’s email, complete with the incriminating evidence.

There it was: logs showing the MAC address of my cable modem being involved in suspicious BitTorrent activities.

Considering that at any time during the week there can be from two to six or seven different teenagers hanging out in my humble abode, carrying virus-ridden machines, the message was clear: I had to get serious about locking down network access

The Problem

I would have liked to have bought some net filtering software to slap on the offending machine and been done with it, however I knew that this was insufficient.

Even if this one event could be traced to a youthful source, a more ominous danger comes from the inevitable malware and viruses that teenagers collect on their machines as they swap cool stuff with their friends.

Complicating things, there are many devices on our home network: Besides their school laptops, the kids have video game consoles and one has an iPod touch, all with wifi access. Think about how many different gadgets are on your home network.

And shutting off access altogether was not an option—there is still schoolwork to be done!

The answer: A Private Network for the Kids

My solution was to put together an unusual network configuration using a second wireless router; I wanted the ability to manage every single kid-owned device at the flip of a switch, while leaving the grownups untouched.

I hooked the cable modem (red) to the main router, shown in green. I then plugged a second wireless router, shown in blue, into the first.

By doing this, you can see that there is one single wire connecting the entire blue network (the kids) to the green network. It was trivial to then configure the green router with appropriate access control and filtering for that one single device: the blue router.

Some quirky details

Home routers like these are, by default, configured with a NAT firewall. They work sort of like one-way mirrors: someone on the network can see out, but nobody can see in. As a result of this, the kids (blue devices) can see any device on the main router (green devices), such as our print server and the NAS device, but no one can see into the kids’ network.

As paradoxical as it seems, this is exactly what I wanted. By making the kids’ network a private network, it appears to the green router as a single device. When I am configuring access restrictions, I only need to control access for the blue router’s IP address or MAC address.

Many consumer-grade routers have flakey firmware that just doesn’t really behave well when you start doing things like turning on filtering for multiple machines. I simplified things by bringing down the number of controlled devices to one. In addition, if one were to try filtering on the IP addresses or MAC addresses of individual machines, this can be easily defeated by manually changing the IP address or MAC address. With my configuration, the MAC address being filtered is the blue router, locked away safely.

The Finer Points

If you want to set up a network like this, do the following:

• (Recommended) Reset the kids’ router. Hold the hard reset button on the router in while you turn on power; hold the button for 15 seconds or so.
• Hook the kids’ router up to a spare laptop using an Ethernet cable. (Turn off the wireless of the laptop for the time being).
• Use the laptop to navigate to the configuration web page (usually 192.168.1.1).
• Set the router’s own address to a different network from the main network, such as 192.168.2.1. This is critical.
• Configure the router’s gateway and DHCP server entries to all point to the main router (192.168.1.1). This tells the kids’ router to use the main router as a source for its DHCP lookups and such, rather than going to cable modem.
• Navigate to the configuration web page at the new address (192.168.2.1). You may need to close the browser and replug the Ethernet cable.
• Set up your wireless security for the kids however you like. Make sure to choose a different channel and SSID from your main router.
• Remove the laptop and plug the WAN port of the kids’ router into one of the LAN ports of the main router. Restart everything.
• Test both networks to make sure things work the way you think they should.
• (Optional) You might want to connect to the kids’ router and set it’s external IP address statically. Make sure that this is set to a number on the home network (e.g. 192.168.1.2).

Some notes:

• You can only maintain the kids’ router from a machine connected to the kids’ network; the home network cannot see the management screens. If you wish, you could enable remote management for the kids’ network only, since the main home router is still protecting the whole network from intruders.
• Computers on the kids’ network can see all devices, but they aren’t on the same network. This means that network printers and NAS devices are accessible, but you will have to attach to them using IP addresses. I was able to easily set up the machines on the 192.168.2.1 network to use a print server on 192.168.1.100.
• For machines that should have full access (a.k.a. yours), make sure that you either set the green network to be a higher priority or remove the blue network SSID entry altogether. I found out the hard way that my iMac would randomly pick the green or the blue depending on which one it saw first when it woke up.
• This does not wall off your main network; it simply provides a single point of control to the entire kids’ network. In other words, don’t depend on this setup to prevent malware on the kids machines from seeing your machine. You can, however, set up your PC to not trust the kids’ network.

Wireless Network Security

Regardless of how you set up your network, make sure you use at least WPA encryption (Never use WEP!). Make sure your passwords are solid.

Using DD-WRT on my new wireless router

In addition to the new network configuration, I went one step further and chose a main router that lends itself well to installation of open-source firmware. I ordered a Linksys WRT54GL from Amazon for a little over fifty bucks. I chose this one because, as a direct descendent of the venerable WRT54G, this router is very well suited for running alternative firmware such as DD-WRT, giving substantial control over things like, say, access control…

Within a half hour after my new router arrived, I had gone to the Supported Hardware page, obtained the latest build of DD-WRT, and replaced the Linksys firmware with the far-better open source code.

I won’t go into the specifics of installation here, but it isn’t very challenging. Check out the DD-WRT site for details.

Closing Thoughts

Make no mistake: we are responsible for whatever goes on our home networks. Just like your home telephone; if someone dials up some 900 number and rings up a thousand-dollar phone bill, the phone company won’t care a whit who did it, you will still pay. Likewise, regardless of who did the BitTorrent download, there is a certain degree of responsibility of the homeowner to lock down the network.

Another point: Without some degree of personal responsibility on the part of the kids in the house, this sort of activity would simply be an arms race of filtering and blocking versus hacking. My goal is to help keep the honest people honest and to make life more difficult for the viruses and malware.