What's your data storage and backup strategy?


#1

I’m currently rethinking my backup/data storage and want to know about yours.

My current setup is pretty dodgy but has worked OK so far:

  • Mac mini server has a 4TB drive connected by USB, this drive holds all our movies, tv shows, photos etc and feeds into Plex Media Server running on the mini. PlexConnect enables a couple of AppleTVs to stream our local content.
  • HP N40L Microserver running NAS4Free with 5x 1TB drives in RAID5, configured as a Time Machine target for the Mac mini and 2x MacBook Airs.

I’ve done it this way around so that the Microserver doesn’t need to be running (for whatever reason it might not be) in order for Plex media, photos etc to be accessed.
I can’t help but feel there has to be a better way, I’m open to spending a little money but you can assume that ~4 TB of storage is sufficient.


How Do You Backup?
What's your current Apple/Mac setup at home (and at work)
#2

Your solution sounds really good to me!

I have 2 x 3TB drives in a single enclosure just left as individual drives and Time Machine alternates backups every hour between the two. The enclosure isn’t on a network, it’s connected directly to the machine via a dock, so when I’m out and about with my laptop it just saves backups locally until I connect to the enclosure again.

Ideally, I want my backup drive on a network like you have and I want a Time Capsule to do it, but those things are a rip off!

The only thing I’d suggest if you’re not doing it already is to backup documents and photos to a cloud provider like OneDrive or Dropbox. You can’t be too careful!


#3

My data redundancy setup:

Data (mainly media and photos) is saved onto a Thunderbolt Promise Pegasus 4 x 1 Tb array in RAID 5.
Home and Work documents are saved onto a 3 Tb Copy cloud account mirrored on multiple computers.
Photos and Videos are saved onto a 1 Tb Dropbox cloud account (which shows up on Carousel similar to the Photo iCloud service but cheaper).
The data on the Pegasus, including Dropbox and Copy, is backed up using Crashplan Pro (unlimited) to 2 spots:
1 offsite in the cloud - Crashplan servers offsite in Sydney.
1 locally on an external Lacie 3 Tb Thunderbolt drive.

That’s my paranoid backup setup. :slight_smile:


#4

I have my media similar to iain (external 4TB HDD connected to Mac Mini). Filching a bit by the seat of my pants, as I have no backup for video files at the moment. iTunes media is on second HDD in my MBP, also not backed up**

My 1st gen Time Capsule died recently, and have a QNAP with 2x2TB drives as replacement for this. Backing up MBP and MBA (Should add Mini to that).

Current documents are saved in iCloud.

**I don’t have anything large enough to back-up my media. I do have some other external drives (and lots of smaller HDD), though it is a bit of a pain to organise. Open to recommendations as well?

How do you set that up? Might work for an offsite backup of my MBA (MBP stays at home as screen no work)


#5

You just need to add the second drive in Settings > Time Machine and that’s its behaviour by default.


#6

Until recently, I had a Synology NAS that backed up to a fire/water/theft resistant ioSafe hard drive. However, last month I replaced the aging Synology with a fire/water/theft resistant ioSafe NAS (it too runs Synology DSM, which is nice) which now backs up to a standard WD hard drive (the ioSafe hard drive was aging too). There are quite a few benefits to this setup. The Synology - and now ioSafe - act as a personal cloud. And, while the ioSafe devices certainly aren’t cheap, they actually work out to be less expensive than backing up using, say, CrashPlan’s Family Plan when you spread the cost over 5 years.


#7

Backup is a pain, even when you’re doing it there are plenty of risks that could/should be addressed. Lately I’ve been re-thinking my strategy because of the potential of crypto-locker like virus’. While I main use Mac at home I do have a few Windows VM’s and I do have a Surface which syncs my main documents folder between the two.

I do wonder what the device list for “normal” people looks like. Maybe something to discuss in another thread… In our house we have a bunch of Macs, some Windows VM’s a Surface which visits as well as a bunch of iOS devices, gaming consoles and media players.

Backup is limited to the active Mac’s with none of the VM’s or the Surface being directly backed up (although there are snapshots of the VM’s, but these are more appliances than actual machines so it isn’t relevant).

The Mac’s use a 2TB TimeMachine, although this has recently hit capacity. I know it’s designed to auto prune, but it keeps popping up a warning and I keep considering deleting all the history and starting fresh for each machine. Sadly there is no built in function to prune a single backup archive to a certain date (that I know of) so it’s all or nothing.

Behind this I have an 8 Bay QNAP with 8 x 4TB running RAID 6 which provides mass storage for:

  • Our DVD collection all lovingly ripped and stored for access via Kodi (I keep trying Plex and it is installed but I just can’t get into it) which is running on a Mac Mini on both my TV’s.
  • Our personal photo/video library.
  • Documents backups (although this is an ad-hoc manual process currently)
  • VM Snapshots.

Plugged into this there is a 4TB USB HDD which replicates the photos and documents, everything else I consider replaceable (and as they are backups of physical media or running VM’s I consider them replaceable and not worth backup up again). This is currently about 1.5TB worth of stuff I want to keep, mostly the family photos/videos.

I used to have a second USB HDD that I’d swap out when I remembered and leave at mums house, but I dropped it and destroyed it one day and never got around to replacing it… then I moved states making it harder again.

This is all sitting in a rack with filtered, UPS protected power.

At this point I feel pretty confident that I’m protected from physical failure. Power spike/flood/fire/theft are still a concern.

So recently I added in some offsite stuff that I’ve automated:

  • My photo’s are all automatically uploaded to Google Photos. (being on the free tier they are compressed, but it’s probably more than enough).
  • My documents are automatically synced into OneDrive.

Even though I moved to a cable area, the shitty upload rates meant that this took a few months to get the initial upload done :frowning: (Bring on FTTP and decent upload speeds I say).

Now I feel OK about major physical issues of the equipment.

BUT, recently I’ve read a few articles about crypto locker and that worries me since my backups are automated and have the potential to sync busted files across everything and kill my whole setup. For that you need multiple offline copies or backups with versioning.
I’ve considered tape, but the setup costs can get high and it becomes a manual process.
USB HDD’s are cheap, but it’s still a manual process to swap them over.

I’d love to setup a backup with versioning, I still have an older NAS sitting under the bed and I’m looking into using it at the other end of the house or in the garage or something. I can automate this and I could set it to auto power up and down to save on power too.
Ideally I would put this NAS at someone else’s house (or access their NAS) for my backups which gives me the best of all options with automated offsite backup with versioning… the limiting factor being upstream bandwidth.
Versioning isn’t a major bandwidth problem since we are talking photo’s that will (or should) never change anyway. Only new photo’s would need syncing unless cryptolocker were to come along and then hopefully you would notice pretty quickly anyway.

Amazon glacier storage is an alternative option, besides the initial upload time it’s about $12 or so per TB / month for storage but almost $150 per TB to retrieve should you ever need a bulk retrieval. Not alot of money in the grand scheme of things (for the storage side of things), but something which will only grow and will add up over time. And that’s before you consider the tin foil hat question about putting your stuff on to the internet (if you’re that way inclined).

Sharing with a friend (where all you need is FTP/SFTP access to a folder (which could be a NAS or a dedicated USB HDD) seems to make more sense to me, although it requires a high degree of trust in that friend (or family member) since you are giving them all your files (encryption aside).


#8

For those interested, Anthony posted a great article on his backup strategy on Reckoner a few weeks ago. It’s a really good read and made me rethink my strategy (even though I thought my backup plan was fairly good).

The article is here http://reckoner.com.au/2015/08/how-i-keep-my-precious-data-safe/


#9

Great article.

I do subscribe to the 3-2-1 principle too (I think).

Local drive is on a RAID setup in case of drive failure.
Local backup to external drive (using local crashplan instead of time capsule).
Everything (iTunes library, documents, personal media) has cloud backup to Crashplan (3 Tb in cloud so far).
2nd cloud backup for personal photos and videos to Dropbox (1 Tb).
2nd cloud backup for documents to Copy.com (25 Gb so far).


#10

I just read Anthony’s write up and will have to look into Amazon Cloud drives unlimited storage as that could be really attractive and help out.

Arq looks like an interesting tool too, I like the idea of encryption (which I haven’t looked at yet). I still want to learn more about how/if it handles versioning… maybe when I get some spare time.


#11

I have a 4TB Western Digital NAS Cloud Drive which, despite fears of non-Apple-HFS compliance, houses my iPhotos, iTunes, and general movie files.

I then have a 2TB drive connected to my Airport Extreme which is the Time Machine drive for my iMac and MacBookPro… Err… which means lots of the Cloud isn’t backed up… Err… That’s not good.


#12

Mines pretty simple compared to a lot of yours.

Data is on a single 4TB external drive. I looked at putting this on a NAS but why I didn’t:

  • Backblaze doesn’t do NAS backups
  • My MBP’s the only device that touches it
  • There’s nothing on there that’s so critical to me that if it dies, I can’t do without while I just replace the drive and restore from backup

Time Machine backing up to my Time Capsule (anything and everything).
Backblaze.
iCloud Photo Library (not BackBlaze) backs up my photos
iTunes Match (not BackBlaze) backs up my music.
Reason BackBlaze doesn’t do my photos or iTunes is there’s just too much data to make it worth chewing up my internet connection for weeks on end. ADSL2’s the best I can get here.

Reason for not having a proper NAS for my backups is again, it’s just not worth. If I’ve lost my Mac and my TC I’ve likely experienced a flood, fire or theft. I could invest in a couple of local drives but for the price of two drives to rotate, BackBlaze will FedEx all my data on one.

Just a couple of notes:

  • A RAID is not a backup. Don’t downplay your backup strategy because your data’s saved on a RAID. RAID is a form of redundancy and availability in the event of drive failure. I’ve known people who’ve thought their data is safe in a RAID, had a drive die, replaced it and then had a second drive die while their RAID was still rebuilding.
  • A good versioning system (ShadowCopy does this in Windows, I believe OS X does this as well but not 100% sure) will backup only the bits that have changed (rather than the creating a new version of the entire file).

#13

I’ve never managed to organise myself well enough to have a NAS or decent backup strategy. I have multiple 2GB HDs and two separate drives for TM and my SuperDuper clone. I used to also make a media drive clone but now I don’t bother, I’ve dumped everything except the stuff I ripped from my own DVDs. The stuff on the drive is the backup. Netflix has seen the end of my collecting things. I don’t have anything that could possibly be considered irreplaceable except photographs, and only a few of those are actually worth the keeping… the ones of friends and family and a few arty-farty shots. For the Macbook, a small USB portable is sufficient. But really… not sure why I bother, because the contents of the macbook arent worth anything. its just a convenience to me and isnt even a repository for pix.

I wanted to get a NAS, not for raid, but just to reduce the number of electrical plugs I have hanging off boards in the study. (Currently far too many!!) I really should do something about that, instead of lusting after a new iphone or ATV.


#14

For cloud backup, I can’t recommend crashplan enough. Cheap monthly fee, unlimited storage, sophisticated versioning and a great iPhone/iPad app that allows you to view our whole drive online anywhere and anytime.


#15

I feel the same way about BackBlaze (If anyone wants a promo code, let me know :stuck_out_tongue: )


#16

I have a not dissimilar strategy to some of you guys.

My main documents are on Dropbox, which is mirrored across a number of computers, so I can continue where I left off if a computer fails.
A Time Capsule backs up my computers, except my iMac which has its own dedicated drive due to the amount of data on it.
Large data such as RAW photos and video and my music library, which can’t fit on the Time Capsule or in Dropbox, are backed up manually (or when I was using Aperture, using archives) to 2-3 other hard drives.
Lastly I have Backblaze backing up my iMac’s main drive.

Some thoughts: I don’t like the idea of using a NAS. I had a WD external drive which, if the hardware died, the internal drive wouldn’t have been readable by anything. This is absolutely no good. A backup needs to be recoverable easily if it fails.
Secondly, backing up to a multiple-drive array adds a greater potential for failure. About the only solution I’d consider would be a RAID 1 or RAID 10 set-up. I’d only consider an array if it were a professional solution for a business and I was rotating the drives with new ones once a year or two.


#17

I like having my superduper clone on an ext drive. It saved my bacon more than once. For other “stuff”, I currently have multiple drives and multiple power boards. This bothers me. I think a NAS which is just set up as a bunch of drives but which is poweres by only one power supply would suit me better. I need to look into it more.


#18

I have a dual drive Synology NAS, Time machine backs up to the same drive from the 1.TB iMac and then from the 128GB MBA. I then have a Superduper clone of each drive on the other drive I make every few weeks. I could have made them RAID but I want them independent of each other. In a perfect world I would get a 4 bay NAS and then have each of those shadowed. I want to upload the superduper bootable copy to somewhere in a cloud but not sure how to seed it with 500GB and growing needed to be synced first. I also have a WD portable drive that I keep at work and bring home once a week to get a TM backup from both macs onto it. Thats my offsite redundancy, but unless I cycle two of them, there is a night each week where everything is in the same room. Have thought about getting my in laws to setup a Synology as well and do a remote backup to there and vice versa.


#19

Nothing as good as everyone here…

Synology RAID-1 Dual Drive array. Manually run an rsync script.

Internet is too slow for any AWS style backups… :frowning:


#20

Okay, now that I’ve set up a new (used) Mac Mini as a home server/desktop, I’ve revised my overall storage and backup strategy, and thought I would share.

EDIT - added photo of my storage setup, showing MacMini with 2 internal drives, Thunderbolt Pegasus RAID and Thunderbolt LaCie:

STORAGE:

The Mac Mini comes with a 256Gb SSD which runs the OSX system files and apps.

The Mac Mini is connected to a Thunderbolt 4x1Tb Promise Pegasus external array, set up now as 4Tb in RAID 0 (I know, there’s no redundancy, but I get speed and space, both of which I need), which stores just over 3 Tb of data consisting of my Photos library, iTunes library, documents, other media and files. Essentially, my data sits here.

BACKUP:

The Mac Mini 256Gb SSD is cloned using Carbon Copy Cloner to an internal 1Tb HDD so my OSX system and apps are in a bootable backup with a backed up recovery partition, that can be used to boot the system if the primary SSD fails. This backup is incremental and done automatically every week.

The external 4Tb data drive is fully backed up to the off-site Crashplan+ cloud, which offers versioning and live backup.

The Photos and iTunes libraries from the external data drive are also rsync via an Automator shell script app on alternate days on a weekly basis through a recurring Calendar event, to a 3Tb Thunderbolt LaCie drive.

The documents on the external data drive, in addition to being backed up to Crashplan+, is also synced to Dropbox Pro, which gets replicated to my MacBook Pro (and the selected documents to my wife’s MacBook).

The above backup strategy so far therefore involves a bootable OS drive backup, off-site cloud data versioning backup, local data backup, and cloud syncing of documents to various devices - so I think it’s as robust as I’m going to get for now. If I had more funds to upgrade the drives on my RAID setup I would go back to a RAID 5 or 10 setup, but my data I think is safe in the event of RAID failure because of both the cloud and local backups.