Swimming in Gigabytes of Images

A Simple Storage Solution for Photographers

One of the most common questions that I get from photo workshop participants is, “My hard drive is filling up, what should I do?” Over the years, I’ve given the same response, which is that it makes sense to either add an extra hard drive internally (if you have the space) devoted to image storage or add external hard drives, and make sure you have all your drives backed up to another device at your house and online via BackBlaze (if you sign up via my link both you and I get a free month).

I still recommend doing either of those, because it’s the easiest solution and requires little IT management. By that I mean that the user doesn’t need to add and manage an entirely new system in order to access their files. The downside is that after you need to add more hard drives, your system can end up being a tangly mess of USB cables.

The NAS vs. DAS Journey

Over the last couple of years, I’ve seen a lot of videos on YouTube recommending a NAS (Network Attached Storage) for photographers. A NAS for photography is basically a second computer system designed to store your images that is attached to your network, wifi or ethernet, instead of your computer.

You access the files through the network connection instead of a direct connection to your computer. The upshot of this is that you can access your files from wherever you are in your house (or world if you set it up that way).

Also, you can setup a RAID (Redundant Array of Inexpensive Disks) system on a NAS, which can make it very fast. A RAID system basically combines multiple hard drives into one drive and stores multiple copies of the data across multiple drives in a way that it can be resistant to a single drive failure. It isn’t considered a backup. It’s more like a single drive with a little extra protection.

That scattering of data on a RAID systems means that the computer can be fed various parts of a photo from multiple different drives at the same time and that increases speed significantly.

A NAS can be a good solution, but once you plunge into the NAS world, you have to overcome several hurdles and end up running into additional computer management.

Because it was being recommended so often and I knew so little, I worked with Riley Bruce, who comes from the computer world in large-scale data storage and has taken a couple of my workshops, to figure out the NAS world. Additionally, I wanted to understand RAID. My conclusion from the conversations that I had with him was that if I wanted to go with a NAS to achieve my goals, I’d need to do a couple of things. My goal was to make my system the fastest that I could and use it at my main computer in my home office. I didn’t care about being able to access my data from different locations in my house, because I only work in my office.

To do that I’d need to do a couple of things:

1. Add a 10Gbps network switcher and be able to set that up and manage it. The switcher would allow my computer to get data both from my ethernet (Internet) and a NAS.

2. Upgrade my computer’s ethernet from 2.5Gbps to 10Gbps.

3. Connect the network switcher directly to my computer or upgrade all my house’s Cat5e cables to Cat6 or above and store the NAS in the room with my service provider’s modem/router and where all the current ethernet cables originate from. The latter would include remodeling (again).

4. Buying a NAS and hard drives, setting it up, and managing it by learning new software.

After thinking about it, I decided that none of that made sense for me. I really don’t like managing computer systems, so I didn’t want to have to do another. My computer system wasn’t designed to handle the 10Gbps connection, so I’d be limited in speed unless upgrading. Because Backblaze charges backups off of a NAS at a different rate than a simple flat fee for hard drives connected to your computer, it was going to cost more for cloud backup. Plus, if I was going to connect something directly to my system, I could use an USB 3.2 Gen 2x2 port that my computer has, which potentially would achieve a transfer rate of 20Gbps. An even better option would be USB4, and I wouldn’t buy a system without that if I was buying today.

My DAS Experience

Because I didn’t want to be a computer system manager with a NAS in addition to doing what I do, I decided to pursue a DAS (Direct Attached Storage). A DAS is basically an external hard drive enclosure that can handle a number of drives. Technically, a single hard drive like you’d buy at Best Buy is a DAS, but for simplicity, I’m going to call a single external hard drive an external hard drive, and I’m going to call an enclosure designed to hold multiple hard drives a DAS. I ended up buying one with space for four drives. I could also setup RAID on it if I wanted to get that extra speed.

The problem was that there were very limited choices at the time, and many of the choices were reviewed as slow even when running a RAID setup. I ended up with an OWC Mercury Elite Pro Quad, which I do not recommend (It seems from the reviews that some people have good luck with the OWC. I didn’t), and four 12TB Seagate IronWolf Pro drives. I also bought a 24TB drive that I installed in my computer to use as a backup drive.

For RAID, OWC offered a software based RAID. I’m not sure how well it works on Mac, but on my PC it was nothing but trouble for me. I could never get it to complete the recommended disk verification processes that you have to do before setting it up with RAID despite trying to work with their “customer support,” so I bought a program called Hard Disk Sentinel Pro to do those steps. OWC has issued a bunch of bugfixes to their RAID software since I used it, but I’m not going to try it again.

Eventually, the OWC was running a RAID5 setup (the RAID number is mumbo-jumbo for the different ways you can set it up. In RAID5 in my system I sacrificed 12TB of storage out of 48TB for redundancy, which meant if one of those drives broke I could still access all my files and just put a new drive in for it to rebuild the RAID system from the remaining three) , and it gave me 36TB of storage, which was great. What that looked l like was a single hard drive on my system represented by a single drive letter. I loved that. With the RAID system, I was getting somewhere around 600MB/s data transfer, which was pretty darn fast.

But, after a power outage, which happens often-ish where I live in Minnesota, I ended up losing data, and the RAID system had to rebuild itself. Luckily, I had all that data backed up on a separate hard drive and in the cloud. The OWC tanked despite being connected to a backup battery system. I was working on my computer at the time, and the computer was fine and stayed on and I was able to continue to work on it using my backup battery system, but the switch from the main power to battery backup glitched the RAID system. If a power outage was going to mess it up, it wasn’t going to work for me.

So, it was back to the drawing board.

RAID Didn’t Work For Me, Now What?

I decided to run all four disks in the OWC DAS as their own individual hard drives. This dropped the speed of the system to about 200-230MB/s. That was a big hit, but I value reliability over speed. I set up the drives as two main drives that I worked off of, and the other two drives became mirrored backups. I still had an 24TB drive that those two main drives are also backed up into. I figured that since I bought it, I might as well use it. So, I now have 2 local backups of each file, plus the file on the main working hard drive, plus the cloud. That’s four copies of each image if you’re counting.

The above setup shows up on my system as four different drive letters, instead of a single drive letter that the RAID setup was giving me. But, there’s still only one cable connected to my system instead of four if I had just bought four external drives from Best Buy or something.

The OWC was also loud, and with all the noise coming from it I didn’t even want to be in my office. It seemed to randomly shutdown and take a long time to spin up the drives again to access data (I worked around that by forcing it to always stay running if it was on).

In the end, I switched to a TerramasterD4-320 External Hard Drive Enclosure. My experience with it has been night and day compared to the OWC. It’s quiet using the same drives. It’s faster even though the read/write speeds are the same. It seems to instantly read and write instead of pausing before it does like the OWC did. It doesn’t just randomly shut down. My drives stay cooler. I don’t notice a high pitched fan noise from it that the OWC had, and so far I haven’t had any problems. If I wanted to I could use a software-based RAID to run RAID on it.

Maybe my experience with RAID would have been much different had I gone the NAS direction with a Synology system, upgraded ethernet, an upgraded computer card, and a switcher. But I’m finding that using a DAS is working fine, except for video editing. It works okay for video editing.

In the future as SSD drives come down in price, I plan on getting a second DAS designed to work with SSD drives or upgrade the SSD drives in my computer and make those my main working drives and then backup to the Terramaster (or whatever I’m using at the time). 8TB SSDs currently run between $650 and $1200, which is more than I want to spend. Currently, if I’m editing video, I edit off of one of the SSDs that I have installed in my computer (3,600 MB/s). After the edit is over, I move the files to an external hard drive. I don’t edit many videos, so it isn’t a big deal for me.

My Recommendations for Most Amateur Photographers

At any rate, my recommendation for photo storage remains the same:

• Buy an external drive or DAS and use that to store your images. Call this your main drive. Make sure it is 7200rpm for speed and at least USB 3.2 Gen 2. Make sure it is bigger than you think you will need. Replace it every couple of years or as needed with a bigger drive.

• Buy a second external drive or use a slot in your DAS to back up all your files from your main drive. I use Syncovery, but I bet there is easier backup software out there. I’d love to get a recommendation for the easiest backup software that I could recommend to people.

• Back up your images to the cloud.

This is simple, and for the majority of amateur photographers, it’s going to be a great solution without a lot of extra computer management. You just need to plug the external drives into your system and they work. With the DAS, you just need to insert the drives into the DAS, plug it in, format the drives, and then it works. You don’t need to worry about extra hardware components or extra software to learn and manage or learning to manage RAID. You can just shoot, load the images onto your computer, and edit away.

I do plan on getting a NAS to experiment with. When I’m making maps I like to be able to work in different rooms in my house and on either of my computers. I’d like the same data to be available on both computers, so a NAS would make sense. I’d also like a NAS to be able to backup the photos on my phones, so I could stop paying Google for extra Google One storage just for phone storage. For that, I want to try a BeeStation, and then backup that BeeStation to my main computer and then to the cloud. But, I’m not motivated that much to try it anytime soon — maybe after my busy workshop season is over towards mid-November.

This is a long newsletter, and to wrap it up I’ll just say that storage is somewhat of a nightmare. The best solution is not keeping all the bad shots. I’m still working on deleting duplicate and bad images in my catalog, and I’m getting much more picky in what images I keep. I also think that manufactures haven’t really given the average photographer a good solution for storage when it comes to speed and simplicity. The best bet, if you are looking, is a pair of external drives. Your main drive and a backup drive.

Until next time

I hope you enjoyed this somewhat rambling deep dive into the mysteries of storage, and I’ll see you next week. Until then, here’s the parting shot.

p.s. my Northern Minnesota Gravel Ride Guide: Cook County Edition is now available as an eBook or a print-on-demand physical copy. Copies will be in local stores soon.