Diagram #3 - Second Drobo Mirrored Backup

Image Management Workflow for the Mobile Photographer (Podcast 466)

Over the years I've developed and evolved a pretty sound file management workflow for working with Lightroom on multiple computers, both in the office and when I'm traveling. I've talked about various aspects of this in previous episodes, but I thought I'd report on my current image management workflow for the mobile photographer. I'm going to explain how I currently manage my Lightroom catalog, settings and presets, and my photographs and video archives, including how I now move from one computer to another quite seamlessly, but first a little background. Until now, I've kept my Lightroom catalog on the internal hard drive of my...

Thank you for visiting! To help us to continue to provide quality posts and photography inspiration we are asking visitors to pledge a Patreon contribution of $3 or more to unlock the full text of more than 750 posts. The Podcast itself is and will always be free, so you are welcome to listen with the above player and follow along with the images discussed below. However, if you value what we do, please consider supporting us with a Patreon pledge.

There are also higher tiers with community access, monthly desktop wallpaper images, eBooks and a monthly Patron-only live Question Time event. There is even a Mentorship program if you want to take your photography to a whole new level. Of course, all tiers unlock all posts, to please click the Patreon button below to view details. Thank you for your support!

Become a Patron!

Existing Patrons please login with this button to unlock posts and access benefits. Thank you for being awesome!

Image Gallery

To view this content, you must be a member of Martin's Patreon at $3 or more
Unlock with Patreon
Martin Bailey
Martin Bailey is a nature and wildlife photographer and educator based in Tokyo. He's a pioneering Podcaster and blogger, and an X-Rite Coloratti member.
34 Comments
  • Jason Underhill (@underhillj)
    Posted at 03:36h, 08 April Reply

    Great Podcast! Thought I’d drop you a comment to let you know about a tool I use which exists on almost every OSX or Unix system out there.

    rsync is a free utility that is command line driven so more suited to technically minded people however it’s a great tool and is probably what ChronoSync is built on top of. So for anyone that doesn’t want to have to pay for a sync tool I can highly recommend it. I use it for keeping my physical backup drive up to date along with syncing my working directory with my archive drive.

    http://linux.die.net/man/1/rsync

    Thanks,
    Jason

    • Martin Bailey
      Posted at 09:21h, 08 April Reply

      I’m sure you’re right Jason. I would imagine ChronoSync uses RSync in the background. Thanks for stopping by and for the recommendation!

  • Allister Freeman
    Posted at 23:25h, 08 April Reply

    Great article Martin, very useful resource

  • Michael Zimmermann
    Posted at 15:34h, 09 April Reply

    Hi Folks,
    my cheep solution for keeping a second copy is a batch file with …. xcopy ‘s …. this copies only newer/changed files and does not delete anything.

    Selected Important Lines.

    @ set DRV05=E:
    @ set DIR05=\
    @ set DRV06=F:
    @ set DIR06=\

    @ set PARM2= /V /C /I /R /Y /H /f /S /D
    REM /V Verify /C Continue /I Destination Directory… /R Overwrite Read /Y Suppresses prompting /H hidden /F full source /S subdirectories /D:m-d-y Specified Date.

    @ set SUBDR=\Share_12
    @ echo %DRV05%%DIR05%%SUBDR% %DRV06%%DIR06%%SUBDR%
    xcopy %DRV05%%DIR05%%SUBDR% %DRV06%%DIR06%%SUBDR% %PARM2% >> %Log%
    @ echo %DRV05%%DIR05%%SUBDR% to %DRV06%%DIR06%%SUBDR% >> %Log%

    • Martin Bailey
      Posted at 15:51h, 09 April Reply

      Cool! Thanks Michael!

      Could you please just recheck the code you posted, to ensure that this should simply be copy and pasted for use? Sometimes comments can mess code like this up.

  • Michael Zimmermann
    Posted at 16:05h, 09 April Reply

    The code extracts left at 15:34h, 09 April are correct, but anyone wishing to use them, should have some basic batch file scripting experience. Experimenting in a Command window helps here.
    Eg.
    @ set Log=C:\Temp\XcopyLog.txt
    xcopy /?
    xcopy E:\Share_12 F:\Share_12 /V /C /I /R /Y /H /f /S /D

    E:\Share_12\Picture5\2015\20150123\Z20150123_130151.CR2 -> F:\Share_12\Picture5\2015\20150123\Z20150123_130151.CR2
    E:\Share_12\Picture5\2015\20150123\Z20150123_130151.JPG -> F:\Share_12\Picture5\2015\20150123\Z20150123_130151.JPG
    E:\Share_12\Picture5\2015\20150123\Z20150123_130151.xmp -> F:\Share_12\Picture5\2015\20150123\Z20150123_130151.xmp

    3 File(s) copied

  • Lionel Thomas (@LioThomas)
    Posted at 19:15h, 09 April Reply

    Nice podcast @martin ! Always inspiring!

  • Vic
    Posted at 04:28h, 14 April Reply

    For Windows may I suggest SyncBack Free. It seems likely this will do everything that Chronosync does. Various backup schemes are supported, including synchronization to keep both file systems the same. And everything can be scheduled to run when you choose, or run manually whenever you like.
    [This is not a paid endorsement and I don’t work for the company.] 😉

  • Derek
    Posted at 19:19h, 14 April Reply

    Another +1 for Vic’s nomination of SyncBack.

  • Leo Cavallini
    Posted at 19:20h, 19 April Reply

    Thanks for sharing Martin!
    I’m into Windows and recommend FreeFileSync. It’s open source and very dynamic. I use it for five years now, I believe.

  • Liam Hammersley
    Posted at 22:56h, 21 April Reply

    I really like GoodSync, which is a cross platform bit of software. I use it a lot to backup my Photo Library to my NAS and an Offsite Server.
    http://www.goodsync.com/

    • Martin Bailey
      Posted at 11:06h, 23 April Reply

      Looks Great Liam! Especially if working between Mac and Windows. Thanks very much for the recommendation!

  • Paul C
    Posted at 19:37h, 28 April Reply

    Hi Guys, I use Bittorrent sync (www.getsync.com) for syncing my data between different machines on my net work, it very fast and is cross platform so will work on Mac/Linux and Windows.

    • Martin Bailey
      Posted at 20:14h, 28 April Reply

      Good recommendation Paul. I’ve used Bittorrent for synching between computers in the past too.

      Can you sync between drives on the same computer with Bittorrent? I don’t remember.

  • Aaron Priest
    Posted at 02:39h, 01 August Reply

    Another +1 for SyncBack. I use Pro. I tested out several notable file syncing programs early last year to see which was fastest and most reliable on Windows 8.1 under a variety of scenarios (thumbdrives, SSDs, NAS, RAID, etc.). I’d been using 2nd Copy and TeraCopy. I tried out GoodSync, FastCopy, RichCopy, Robocopy, RSync, xcopy, EMCopy, and many others. In the end I settled with SyncBackPro because it was multi-threaded, it pre-allocated files properly for less fragmentation, and it had the most options for scheduling, two-way syncing, mirroring, backing up (preserving deletions), etc. And this isn’t a paid endorsement either, I’m just very pleased with the performance and reliability over the past 18 months or so.

    Great topic and podcast Martin! I do things a little differently for my needs. I run my complete catalog on my striped SSD on my workstation, with the image files on a large internal RAID array. Everything gets backed up every so many hours to my Synology. When my main workstation goes down (hard drive crash, motherboard fried, whatever), I can copy the backed up catalog from the Synology to the SSD on my laptop and still access all my files over the LAN to my Synology. I just right click the top level folder and tell it where to find the photos and work off my NAS. It’s slow, but it works in a pinch. I also do this for workshops on the road where I might want to access an old file for reference or for a client when I’m gone for a few weeks. It’s hardly a mobile solution of course, as the Synology is large and requires AC power, but it works for teaching workshops in a classroom.

    When shooting and editing on the road, I use a separate catalog file on my laptop that I import later to my main catalog back home on my workstation. I have an external USB 3.0 hard drive for this as several nights of timelapses won’t fit on my laptop’s SSD. I’ve been considering the Drobo Mini for this for more redundancy and speed. I still have two external USB 3.0 drives that I keep in sync though for redundancy/backup, or I back everything up to the larger Synology if it’s with me.

    • Martin Bailey
      Posted at 12:16h, 01 August Reply

      Hey Aaron, thanks for the detailed information!

      That’s a great endorsement for SyncBack Pro. You did a lot of great testing there. Thanks for providing that list. It’s going to be very useful for anyone looking into this kind of think. Did you do a blog post about it? Drop a link in if you did. I’m sure others would also like to this this.

      I know it’s only been a few months, but I’ve actually stopped using my Drobo Mini for this workflow. Having to plug it into the power started to become very tedious, and it’s just too big for travel, especially air travel. When I first looked around for a fast external and portable hard drive, there were really not many alternatives, but I recently found the WD My Passport Pro, which is a 4TB Thunderbolt (only) drive. I’ve ended up buying two, one as a backup of the first, and with 4TB I will be able to fit plenty of images in there while traveling.

      It’s great to have this amount of storage, powered from the laptop, and actually slightly faster than the Drobo Mini. I guess because it’s Thunderbolt though, that kind of rules this out for Windows. I hope someone develops a board or some kind of hard wear bridge for Windows at some point.

      • Aaron Priest
        Posted at 11:28h, 02 August Reply

        I didn’t hang onto all of my speed tests and screenshots to write a good blog post about it. I wish I had.

        I hear you about the Drobo Mini. I use a 1TB 2.5″ USB 3.0 powered drive for simple things on the road (no AC power), but I run out of space on it quick, and there’s no redundancy without cloning it to another.

        So via Thunderbolt you can power the drive just like USB 3.0, no need for external power huh? At 4TB, it must be a desktop drive and not a notebook sized one? That must drain the laptop battery pretty quickly?

        Windows supports Thunderbolt, but driver support by Intel hasn’t been real great yet. Windows 10 will probably change that with native support by Microsoft. It’s hard to find a decent laptop with a Thunderbolt port though (that is still a portable laptop and not a desktop replacement, LOL!). I’m hoping USB 3.1 changes all this soon.

        • Martin Bailey
          Posted at 08:57h, 03 August Reply

          Hi Aaron,

          Yes, the redundancy is why I always buy these portable drives in pairs.

          The WD My Passport Pro is not a desktop drive. It’s a 2.5″ side, with a bit more height, as it has two 2TB 2.5″ drives inside. It’s actually a stripe, and can also be configured as a 2TB mirrored drive, if redundancy in a single drive is important to you. See here: https://mbp.ac/wd4tb

          It’s not a huge drain the laptop battery either.

          That’s good that Thunderbolt is coming to Windows.

          Cheers,
          Martin.

  • afotografy2013
    Posted at 02:40h, 26 December Reply

    Hi Martin,
    I’m middle of massive backup system workflow.

    I like simplicity of yours. I have seen soooo many complex ones that I’m a bit going crazy.

    At the moment I have 4TB WD attached to main desktop. I also copy my cards first on my main 4TB HD and then I use LR to edit from it.

    I have filled up that HD now and not sure what should be my main next move. First of all I need to back up my 4TB one as so far I back up only final selection for clients on separate drive. So I’m thinking to get one more 4TB drive to back it up.

    Now I’m wondering if I should get some sort of drobo system instead of many desktop HD’s.

    I don’t do that much traveling so that is not an issue for me.

    My head spinning of what to back up and what not and what to use for each of these steps. Madness.

    Need to get something in place.

    Thank you and will read through this few more times 🙂

    • Martin Bailey
      Posted at 03:33h, 26 December Reply

      You definitely need to back up your 4TB drive. It will fail at some point, so unless you do not need all of the photos on there that are not your final selects, definitely keep this in mind as you consider your strategy.

      A Drobo would be a good way to store your images, although they can be expensive. a Drobo 5D starting with say two 8TB WD Red drives would give you 8TB of redundant storage, so you could copy your current two 4TB drives to that and make the Drobo your main storage, and continue to use the two 4TB drives as a backup perhaps.

      Then, as you need more space, you could add up to three more 8TB drives, to keep all of your images on one drive. Personally, I use a second Drobo 5D as a backup of my main one, but if necessary, multiple other external drives would work too.

      Also consider Backblaze as an addition cloud backup service. It’s best to have something offsite as well as your local (home) backups.

      • afotografy2013
        Posted at 07:39h, 26 December Reply

        Hi Martin,

        Thank you for the reply. I have spent a day looking through various options and comparisons. Seems like many photographers go for Drobo and few other big brands, but I will be a bit short on my budget to get Drobo enclosure alone which costs £600 plus and drives one top of that.

        What I was thinking for now is maybe to get like good WD internal drives (like the ones you use in Drobo) and buy them casing boxes. Then when I have extra cash, take them out of casings and put them in enclosure unit.

        I understand importance of back up, but need to wait around 6 month to get all properly set. Just got new equipment upgrade.

        Also one more questions.

        See your drobos, are they linked to LR as well or not?

        While I was researching I came across very interesting article on Fstopper where they were talking about having “editing” fast hard drive like 7200rmp or SSD, use that for LR catalog/editing and then have 2 back up systems for archival. The interesting part was that they initially use editing drive to get everything edited in LR and then export each job as catalog to back up archival drive. Then they delete entire job from editing hard drive. Making sure that edit HD doesn’t fill up. Have never exported jobs as catalogues, but this could be option maybe to avoid my main HD filling up? Is this how you do it too?

        Thanks Martin.
        Regards,
        Armands

        • Martin Bailey
          Posted at 01:58h, 28 December Reply

          Hi Armands,

          I actually no longer use Lightroom, having jumped ship to Capture One Pro, but I do reference all previous year archives directly from the Drobo. I reference the current year and my Finals from all years to date on an external SSD drive that I’ve started using recently, because I move this drive between my iMac and MacBook Pro constantly. If I only used one computer, the Drobo via Thunderbolt would be fast enough.

          If the enclosures that you are thinking of are only USB 3.0 then it may not be fast enough for stress-free image editing. You’d need to keep your current year or at least recent shots on an internal hard drive. You don’t have to export images as catalogs. You can move them to your external drives and keep them in the same catalog as your internal drive images. Lightroom can reference images on many different drives, and because Lightroom works fine with very large image libraries, I would recommend not splitting up your images into multiple catalogs.

          All you need to do is move your first batch of images manually, then when you see the question mark against the folders in Lightroom, right click the folder and relocate it. Then show the parent folder, and you could then just drag any other folders to the new hard drive inside Lightroom, and it will just keep track of everything for you.

          I hope that helps!

          Regards,
          Martin.

      • afotografy2013
        Posted at 07:59h, 26 December Reply

        Oh and one more question about Bacblaze. I just had a chat with them and since I’m a photographer they said the best would be B2 Cloud storage which costs around $150 per month. Do you have B2 or Business back up per computer for $50 per year?

        I wasn’t sure what means per computer. Could it be per HD?

        Thank you Martin.

        Regards,
        Armands

        • Martin Bailey
          Posted at 02:01h, 28 December Reply

          Hi Armands,

          I’m sure they’d prefer you to do that, but their $50/year plans are unlimited storage, and there is nothing to stop you using them. I use a $50/year plan and do not need any of the features of their B2 plan, so I’ve no plans to upgrade.

          No, per computer means all drives attached to your computer. I have 9TB of data on Backblaze saved from my Drobo 5D via my iMac.

          Regards,
          Martin.

          • afotografy2013
            Posted at 02:36h, 28 December Reply

            Thank you Martin for reply. My head is spinning. If money would not be an issue I would know what to do, but here I need to play with it a bit. So I’m option out for similar set up as your.

            I’m thinking to get 500GB solid state drive and put LR catalogue on it and run LR from it too. This would be externally attached. I have pretty old iMac 2009 and drive has failed already twice so I’m a bit worried to put anything on actually machine. I have run everything externally since.

            On one side of machine have work station 2 x 2TB fast internal 3.5″ drives and I would load images on them from memory cards. I would set up mirror option with software you were recommending so I have a back up of that drive.

            on other side of machine ( where you have drobos) I would have archive set up. I still don’t know how big these should be but I think for now I will get 2 x 4TB on mirror basis. Also external.

            This is only for my new work as of 2017.

            Not sure what to do with my current work. I have already 8TB of that. Should I just get few drives and copy work on them and put them in foam case. Half of that I pretty much don’t look at anyway.

            Need to get this sorted. I saw someone selling Drobo mini for £100 and was thinking maybe to get that one as my workstation. it has 4 x 1TB drives.

            What do you think Martin.

            Sorry for all long winded story 🙁 scared of worse case scenario.

            Thank you,
            Regards,
            Armands

          • afotografy2013
            Posted at 06:02h, 28 December Reply

            I just realised that its no point for me to have usb3 it will still be usb2 as my mac has only USB connections. Bugger 🙁 so using external HDD will be still USB 2.

            The only option is to change main internal HD then to SSD one. It is possible to do it, checked it.

            I didn’t see that coming. So now it looks like I might need whole new desktop machine if I want efficiently back up. not good 🙁

            Thanks for all of your help Martin.

            • Martin Bailey
              Posted at 10:32h, 30 December Reply

              Oh, I see. That does suck a little. I would imagine a new iMac would be the best way to start, but if you are running out of space, maybe just ensure that you buy something external that will give you the best speed once you’ve upgraded your iMac.

              I’m now using a SanDisk Extreme 900 Portable SSD for my Capture One Catalogs, Finals and current year’s images. These are costly, but very, very fast. http://amzn.to/2idDqoh

              What you’d need to do is ensure that you get something that is USB 3.1 Gen 2. Gen 2 is twice as fast as USB 3.1. It’s basically Thunderbolt 3, and will be used on all new Macs for a while, although these are currently not in the iMac. I’m sure they’ll be in the next generation.

              My point is though, if you buy a fast external drive at this point, consider what will be available when you upgrade, and you’ll future-proof your purchase. Also, I’m plugging my Extreme 900 SSD into a USB 3 port on my iMac, and it’s still fast enough to run my Capture One Catalogs and images without stress. I only get the full speed on my new MacBook Pro with USB 3.1 Gen 2 ports.

              • Aaron Priest
                Posted at 13:55h, 30 December Reply

                I bought a SanDisk Extreme 900 1.92TB USB 3.1 Gen 2 SSD (what a mouth full!) for my new laptop. I can’t say enough good things about it! It’s very well built, doesn’t get hot, and runs well on USB 1.1 (slow of course), 2.0, 3.0, and especially 3.1 Gen 2. Some devices aren’t as backwards compatible, but that SanDisk works on everything I’ve plugged it into with no power issues. It has two mSATA SSDs striped as RAID 0 internally, but it still supports TRIM from the OS. There’s no access to the internal controller, you can’t change RAID levels or tune it, but the average consumer doesn’t need to. I’ve looked high and low for a dual slot PCIe NVMe to USB 3.1 Gen 2 adapter to build my own for even more speed, but I’ve only found a single Chinese manufacturer that makes one, and it’s so slow and unreliable that my old SATA to USB 3.0 enclosure is better. So, in the meantime, I’m very happy with the fast and reliable Extreme 900. SanDisk knocked it out of the park with this one!

                • Martin Bailey
                  Posted at 18:22h, 30 December Reply

                  I couldn’t agree more Aaron!

                  I got mine to go with the new MacBook Pro 13″ and my tests via USB 3.1 Gen 2 have given me amazing results.

                  As I mentioned in the above comment, I’ve also found it fast enough to run Capture One catalogs via USB 3 on my iMac, and that wasn’t possible on my WD 4TB Passport Pro via Thunderbolt 1.

                  Like you say, built, heat management, everything, it’s a huge thumbs up for this SSD.

                  All the best for 2017!

                  Regards,
                  Martin.

Post A Reply to afotografy2013 Cancel Reply