Scan File Management - What to keep, what to toss out

Discuss all BIM related issues here.
Post Reply
sheartab
I have made 60-70 posts
I have made 60-70 posts
Posts: 61
Joined: Tue Oct 03, 2017 2:34 pm
6
Full Name: Sheartab
Company Details: Arch Eng Design Firm
Company Position Title: BIM Manager
Country: US
Linkedin Profile: No
Has thanked: 11 times
Been thanked: 6 times

Scan File Management - What to keep, what to toss out

Post by sheartab »

What are some rules of thumb that you are using for file management? Most of my projects that I scan are large facilities that can range from 500 to 1000 individual scans.

At the moment, I use a Faro S150 scanner to Scene, then export to Recap, then export a consolidated RCP for linking into Revit. After all this is completed, I've got a folder that is HUGE.

Is there a reason to keep the original FLS files if I have a completed SCENE project and Recap project? I've listed one of my folders below. There were 250 scan points for one portion of a building. I assembled them in SCENE first, then exported to Recap.

1. Original Scans (250 fls scans @ 10GB)
2. SCENE Project
3. Recap project (250 rcs files @ 50GB)
4. Recap consolidated project for Revit (1 rcs file @ 15GB)


I don't want to delete anything that I might need later, but I can't say for certain that i would ever need the individual scans again if i have the completed point cloud? I just purchased an 4TB QNAP storage unit in December. Its half full already. :shock:
jamesworrell
V.I.P Member
V.I.P Member
Posts: 537
Joined: Mon Jun 16, 2014 1:45 pm
9
Full Name: James Worrell
Company Details: Bennett and Francis
Company Position Title: Director
Country: Australia
Linkedin Profile: Yes
Location: Brisbane, Queensland, Australia
Has thanked: 14 times
Been thanked: 87 times
Contact:

Re: Scan File Management - What to keep, what to toss out

Post by jamesworrell »

We keep raw scans .. as surveyors we tend to think of these as part of our file notes almost. We are legally required to keep records.

The raw scans have proved handy on occaision.

We keep a cyclone database. Currently zipped in folders by version .. so in theory we can roll versions forward in bulk easier when the time comes.

We also keep an E57 .. as a generic file format we might be stuck opening a proprietary format in 30 years say.

We use Azure for archive ..
tbwester
V.I.P Member
V.I.P Member
Posts: 904
Joined: Fri Apr 20, 2012 1:49 pm
11
Full Name: Thad Wester
Company Details: Clarity Scanning
Company Position Title: President
Country: USA
Linkedin Profile: Yes
Location: SC
Has thanked: 1 time
Been thanked: 2 times

Re: Scan File Management - What to keep, what to toss out

Post by tbwester »

We keep it all. Probably overkill, but after the effort of travel/scanning/reg - seems a shame to delete it!

We bought a NAS, which is 20TB and counting. Also upload it all to google drive for long term storage.
Rikore
V.I.P Member
V.I.P Member
Posts: 353
Joined: Sun Aug 16, 2015 6:48 pm
8
Full Name: Bruce Yager
Company Details: Rikore Geomatics
Company Position Title: Department Head
Country: USA
Linkedin Profile: No
Been thanked: 7 times

Re: Scan File Management - What to keep, what to toss out

Post by Rikore »

Generally we back everything up including the raw scan files. Our main backup is our servers. However, for a reasonable cost we us 8TB Seagate Backup Plus Hubs. B&H Photo sells them for $160. A lot of our projects run close to a TB. We find it a cheep way to keep everything. However you can quickly have a few dozen laying around.
User avatar
LPaulCook
V.I.P Member
V.I.P Member
Posts: 485
Joined: Sat Jul 18, 2015 9:42 pm
8
Full Name: L Paul Cook
Company Details: Land surveyor doing LiDAR
Company Position Title: President
Country: United States
Linkedin Profile: Yes
Location: Santa Barbara CA
Has thanked: 47 times
Been thanked: 44 times
Contact:

Re: Scan File Management - What to keep, what to toss out

Post by LPaulCook »

I tend to keep more than i need. I store all raw files on a separate external HD than the registered scans "just in case. " ive had registered scan folders crash so i want the abality to put them back together again if need be. I love backups! Its important to be able to find old jobs in the future so every job is filed on my external or internal HDs by a job number and street name and client name. A search by any one of the 3 will find the correct job or a short list to look at next. I just bouggt a pare of 10TB internal drives and an external boot item to clone (not just copy) one drive to the other as a backup. I also use Carbonate home to backup everything unlimited data on the cloud for about $60 a YEAR! Why would you not do that! .? Encrypted when it leaves your computer and again when they get it then backed up in multiple places across the world. Just in case.... but thats just me....
L. Paul Cook, PLS
www.LPC3D.com
brian715
V.I.P Member
V.I.P Member
Posts: 199
Joined: Wed Apr 06, 2016 3:24 pm
7
Full Name: Brian Boehmer
Company Details: Meridian Eng.
Company Position Title: LiDAR Technician
Country: United States
Linkedin Profile: No
Has thanked: 14 times
Been thanked: 5 times

Re: Scan File Management - What to keep, what to toss out

Post by brian715 »

We also keep everything. Right now we have 5 sets of external hard drives some are 4tb and some are 8tb. They are sets because one stays in the office and one offsite. The only thing we zip is the original .fls files that are unregistered. a recent 3000 scan project took nearly and entire 8tb drive after scene projects, recap, edgewise, and revit files. For $150 each for an 8tb external, it's much cheaper than redoing your work.
Oatfedgoat
Honorary Member
Honorary Member
Posts: 1047
Joined: Mon Oct 06, 2008 8:31 pm
15
Full Name: Matthew McCarter
Company Details: Costain
Company Position Title: BIM Manager
Country: England
Linkedin Profile: Yes
Has thanked: 5 times
Been thanked: 4 times

Re: Scan File Management - What to keep, what to toss out

Post by Oatfedgoat »

Firstly check what your contracts say about data retention for your clients, this may dictate some of what you do.

I agree with others in the thread, the more data is saved and the more redundancy you have built into your processes through backup and archive the better.

My preference is something along the lines of this:

1. Raw Scan data - Copy this immediately to both your working PC/Laptop/usb drive/network drive for working with. AND
2. Archive copy of raw scan data for disaster recovery (+control txt file if needed) on a secure data source which is in a physical different location to the rest of your data. Now days the cloud is good for this or a satellite office if you have one.
3. Finished registered dataset to be backed up once the lead surveyor is happy that is completed and are happy with it. (In the mean time keep copies of this for working on whilst the project is live in what ever way works for you)
4. Create any interoperable files if needed to translate to other formats such as .pod or .rcp. These e57/pts/ptx/las files could be archived if you wish as these will likely be the most robust and sure way to retrieved the data years in the future after version updates to software make any other option a nightmare)
5. Create .rcp/.pod etc or convert from the e57/pts/ptx/las step 4. NOTE: If a client contract states that you need to retain your data, this is likely what is covered by that part of the contract.

If data needs to be unified, carry that out in step 3 or after step 5

Summary,

Raw x 2 backups, one offsite
Registered x 2 backups, one offsite
Interoperable (optional)
Client required format (optional)
Post Reply

Return to “Building Information Modeling [BIM]”