Anonymize faces, license plates in E57 photos
-
- I have made <0 posts
- Posts: 9
- Joined: Fri Oct 26, 2018 2:46 pm
- 5
- Full Name: Philipp Mielke
- Company Details: STRABAG
- Company Position Title: STRABAG
- Country: Deutschland
Anonymize faces, license plates in E57 photos
Hi all,
more and more companies (especially large ones with own legal departments) are super-sensitive regarding data protection and privacy. At the same time the demand for web viewers to share scan data increases. So we currently face the problem that we somehow have to anonymize recognizable information such as faces and license plates from the generated photos before we can share the data with clients, partners, etc. Obviously, manually checking and editing the point cloud (or photos) of hundreds of scan positions is not an option.
Does anyone have an idea how this could be solved automatically? Any services that can anonymize E57 files? Any workflows?
more and more companies (especially large ones with own legal departments) are super-sensitive regarding data protection and privacy. At the same time the demand for web viewers to share scan data increases. So we currently face the problem that we somehow have to anonymize recognizable information such as faces and license plates from the generated photos before we can share the data with clients, partners, etc. Obviously, manually checking and editing the point cloud (or photos) of hundreds of scan positions is not an option.
Does anyone have an idea how this could be solved automatically? Any services that can anonymize E57 files? Any workflows?
- smacl
- Global Moderator
- Posts: 1409
- Joined: Tue Jan 25, 2011 5:12 pm
- 13
- Full Name: Shane MacLaughlin
- Company Details: Atlas Computers Ltd
- Company Position Title: Managing Director
- Country: Ireland
- Linkedin Profile: Yes
- Location: Ireland
- Has thanked: 627 times
- Been thanked: 657 times
- Contact:
Re: Anonymize faces, license plates in E57 photos
Hi Philipp,Phips wrote: ↑Wed Feb 10, 2021 3:06 pm Hi all,
more and more companies (especially large ones with own legal departments) are super-sensitive regarding data protection and privacy. At the same time the demand for web viewers to share scan data increases. So we currently face the problem that we somehow have to anonymize recognizable information such as faces and license plates from the generated photos before we can share the data with clients, partners, etc. Obviously, manually checking and editing the point cloud (or photos) of hundreds of scan positions is not an option.
Does anyone have an idea how this could be solved automatically? Any services that can anonymize E57 files? Any workflows?
At a guess, you would have to anonymize the photography prior to colourizing the point cloud, or anonymize and then re-colour (and also mask intensity for certain areas). There are a number of applications and services to do this for the photophraphy, the trick is transferring this back to the point cloud. As a minimum you need a routine to accurately transform from scan coordinates into picture coordinates for checking purposes.
I've done this in the past for Pegasus 2 but not had much luck with Pegasus ultimate. You may also need an anonymize that will work with spherical photgraphy. The workflow comes down to the equipment and software that you're using,
Shane
- Leandre Robitaille
- V.I.P Member
- Posts: 542
- Joined: Sat Aug 03, 2019 1:53 am
- 4
- Full Name: Leandre Robitaille
- Company Details: Cima+
- Company Position Title: Civil Technician - Surveyor
- Country: Canada
- Has thanked: 61 times
- Been thanked: 246 times
Re: Anonymize faces, license plates in E57 photos
I have never done this but this is how I would approach the issue. First I would import that e57 in scene 5.5 to conserve the 360 images. Then right click each scans, Operations, color/picture, export color overlay. You would then have all the 360 images in a folder. I would then learn how to use Anonymizer, an AI powered software I know nothing about, but seams to work for that exact purpose.
https://forum.mapillary.com/t/anonymize ... pload/4230 (in this forum they say they corrected 360 images with this software)
https://github.com/understand-ai/anonymizer
Then this is the manual labor again, you would have to manually replace all the 360 panoramic for all the scans. Rigth click, Operations, color/picture, replace color overlay.
This might not be the fastest way, someone might know a faster way to extract/replace the 360images (maybe in cloudcompare? never tried), but hopefully the anonymizer software is a step foward in the right direction for what you want to do.
Good luck ! (keep us updated)
https://forum.mapillary.com/t/anonymize ... pload/4230 (in this forum they say they corrected 360 images with this software)
https://github.com/understand-ai/anonymizer
Then this is the manual labor again, you would have to manually replace all the 360 panoramic for all the scans. Rigth click, Operations, color/picture, replace color overlay.
This might not be the fastest way, someone might know a faster way to extract/replace the 360images (maybe in cloudcompare? never tried), but hopefully the anonymizer software is a step foward in the right direction for what you want to do.
Good luck ! (keep us updated)
-
- I have made <0 posts
- Posts: 9
- Joined: Fri Oct 26, 2018 2:46 pm
- 5
- Full Name: Philipp Mielke
- Company Details: STRABAG
- Company Position Title: STRABAG
- Country: Deutschland
Re: Anonymize faces, license plates in E57 photos
Anonymizing the photos is not a problem, there are services that offer this (e.g. https://www.celantur.com/).
The big problem seems to be finding an automated workflow that extracts all the photos from the E57 and writes them back in once they have been blurred. Doing it manually is not really an option. Maybe the scanner-manufacturers have a solution on how to access the data...
The big problem seems to be finding an automated workflow that extracts all the photos from the E57 and writes them back in once they have been blurred. Doing it manually is not really an option. Maybe the scanner-manufacturers have a solution on how to access the data...
- smacl
- Global Moderator
- Posts: 1409
- Joined: Tue Jan 25, 2011 5:12 pm
- 13
- Full Name: Shane MacLaughlin
- Company Details: Atlas Computers Ltd
- Company Position Title: Managing Director
- Country: Ireland
- Linkedin Profile: Yes
- Location: Ireland
- Has thanked: 627 times
- Been thanked: 657 times
- Contact:
Re: Anonymize faces, license plates in E57 photos
Hi Philipp,Phips wrote: ↑Thu Feb 11, 2021 7:11 am Anonymizing the photos is not a problem, there are services that offer this (e.g. https://www.celantur.com/).
The big problem seems to be finding an automated workflow that extracts all the photos from the E57 and writes them back in once they have been blurred. Doing it manually is not really an option. Maybe the scanner-manufacturers have a solution on how to access the data...
Do you just need to replace the photos with a matching set or are you also looking to update the corresponding scan attributes (e.g. RGB and intensity) based on the changes? Faces are unlikely to be an issue in th scan data but number plates could well be readable from intensity for high density scans. If it is just the former, it should be straightforward enough to develop a solution, either in-house if you have developers available or using a 3rd party, using the E57 SDK. This is something we could do for you if you're looking for options. Re-collouring the scan and masking intensities for blurred areas is more involved and depends on having the correspondance information and algorithms to match scan points to photography. While this might be present in the E57 data the chances are that it is either not there or not suitable, particularly for mobile data. From memory, I think Leica Map factory may offer this as an option, but you'd need to talk to them. Antonio Mendes would be a could guy to contact. Not sure about other manufacturers,
Shane
-
- I have made <0 posts
- Posts: 9
- Joined: Mon Sep 30, 2019 4:10 pm
- 4
- Full Name: Cesar Almeida
- Company Details: Leica Geosystems
- Company Position Title: Reality Capture Specialist
- Country: UK
- Linkedin Profile: Yes
- Has thanked: 1 time
- Been thanked: 1 time
Re: Anonymize faces, license plates in E57 photos
Hi Shane. You are correct. From a development point of view, it's not as simple as just blurring images and importing them back. We have to think about the position, orientation and potential distortion of the images, as the metadata that needs to keep it's integrity over the process of blurring or masking. Then there has to be a process to automatically blur them, and then apply them to the colorization of the point cloud. Accurately and reliably.smacl wrote: ↑Thu Feb 11, 2021 8:27 amHi Philipp,Phips wrote: ↑Thu Feb 11, 2021 7:11 am Anonymizing the photos is not a problem, there are services that offer this (e.g. https://www.celantur.com/).
The big problem seems to be finding an automated workflow that extracts all the photos from the E57 and writes them back in once they have been blurred. Doing it manually is not really an option. Maybe the scanner-manufacturers have a solution on how to access the data...
Do you just need to replace the photos with a matching set or are you also looking to update the corresponding scan attributes (e.g. RGB and intensity) based on the changes? Faces are unlikely to be an issue in th scan data but number plates could well be readable from intensity for high density scans. If it is just the former, it should be straightforward enough to develop a solution, either in-house if you have developers available or using a 3rd party, using the E57 SDK. This is something we could do for you if you're looking for options. Re-collouring the scan and masking intensities for blurred areas is more involved and depends on having the correspondance information and algorithms to match scan points to photography. While this might be present in the E57 data the chances are that it is either not there or not suitable, particularly for mobile data. From memory, I think Leica Map factory may offer this as an option, but you'd need to talk to them. Antonio Mendes would be a could guy to contact. Not sure about other manufacturers,
Shane
So this is something that the manufacturers should be looking at more closely. Sensor integration is the key. How different sensors work together across the whole workflow.
From a Leica perspective, we currently support the blurring of images on the Pegasus 2 This is an machine learning development that we have put in place in the last couple of years. This will blur peoples faces and license plates automatically from the imagery of the Pegasus.
-
- I have made <0 posts
- Posts: 9
- Joined: Fri Oct 26, 2018 2:46 pm
- 5
- Full Name: Philipp Mielke
- Company Details: STRABAG
- Company Position Title: STRABAG
- Country: Deutschland
Re: Anonymize faces, license plates in E57 photos
I think blurring the panorama images would be fine for now.
While blurring the point cloud would be nice, it's probably more of a long-term objective that should indeed be taken care of by manufacturers. Especially, when thinking of intensity or geometrical information...
Anyway, I've noticed that I'm not familiar how a standard panorama image from a laserscan is generated. Is it a) stitched directly from the captured photos or is it b) a projection of the colorized point cloud? I found contrasting claims.
If it's a), could it be an option to blur the photos before they are used for colorization and panorama generation? As a side effect, the point cloud would be anonymized, too. At least the RGB part.
If it's b), could the generated panorama be extracted, blurred and replaced?
While blurring the point cloud would be nice, it's probably more of a long-term objective that should indeed be taken care of by manufacturers. Especially, when thinking of intensity or geometrical information...
Anyway, I've noticed that I'm not familiar how a standard panorama image from a laserscan is generated. Is it a) stitched directly from the captured photos or is it b) a projection of the colorized point cloud? I found contrasting claims.
If it's a), could it be an option to blur the photos before they are used for colorization and panorama generation? As a side effect, the point cloud would be anonymized, too. At least the RGB part.
If it's b), could the generated panorama be extracted, blurred and replaced?
-
- I have made <0 posts
- Posts: 9
- Joined: Mon Sep 30, 2019 4:10 pm
- 4
- Full Name: Cesar Almeida
- Company Details: Leica Geosystems
- Company Position Title: Reality Capture Specialist
- Country: UK
- Linkedin Profile: Yes
- Has thanked: 1 time
- Been thanked: 1 time
Re: Anonymize faces, license plates in E57 photos
Hi.Phips wrote: ↑Thu Feb 11, 2021 10:43 am I think blurring the panorama images would be fine for now.
While blurring the point cloud would be nice, it's probably more of a long-term objective that should indeed be taken care of by manufacturers. Especially, when thinking of intensity or geometrical information...
Anyway, I've noticed that I'm not familiar how a standard panorama image from a laserscan is generated. Is it a) stitched directly from the captured photos or is it b) a projection of the colorized point cloud? I found contrasting claims.
If it's a), could it be an option to blur the photos before they are used for colorization and panorama generation? As a side effect, the point cloud would be anonymized, too. At least the RGB part.
If it's b), could the generated panorama be extracted, blurred and replaced?
A panorama image (forget about scanning for a moment), is essencially generated by stiching images together. Go around in 360 degrees, with overlap, and then, depending on the software, stich them together using common points (matching the same pixels in areas of overlap), or automatically. Different softwares will offer some filtering, blending options to improve\remove the blending\stiching effect and contrast\exposure balance.
for your b) scenario, I think it's doable, but I don't see a real application for it. I mean, if a point cloud is colorized, there has to be images to colorize it, so to take that information from the colorized point cloud just seems redundant. So probably possible, arguably not the best course of action.
For scenario a), it's possible for many years in the terrestrial scanning world to generate panoramic images from cameras, edit them(however you want to edit them, including blurring if the photo editing software allows it), and them match them to a point cloud in the registration software. Leica support this workflow with the Cyclone register product.
- smacl
- Global Moderator
- Posts: 1409
- Joined: Tue Jan 25, 2011 5:12 pm
- 13
- Full Name: Shane MacLaughlin
- Company Details: Atlas Computers Ltd
- Company Position Title: Managing Director
- Country: Ireland
- Linkedin Profile: Yes
- Location: Ireland
- Has thanked: 627 times
- Been thanked: 657 times
- Contact:
Re: Anonymize faces, license plates in E57 photos
The point cloud colours are always derived from the photography but what format that photography will be in can vary based on the equipment vendor, the flat unstitched photos may not be available and re-stitching might not be available. If you're outsourcing the blurring it would be worth asking whether they can support spherical or panoramic photography.Phips wrote: ↑Thu Feb 11, 2021 10:43 amAnyway, I've noticed that I'm not familiar how a standard panorama image from a laserscan is generated. Is it a) stitched directly from the captured photos or is it b) a projection of the colorized point cloud? I found contrasting claims.
If it's a), could it be an option to blur the photos before they are used for colorization and panorama generation? As a side effect, the point cloud would be anonymized, too. At least the RGB part.
If it's b), could the generated panorama be extracted, blurred and replaced?
-
- I have made <0 posts
- Posts: 5
- Joined: Tue May 04, 2021 12:43 pm
- 2
- Full Name: Mario Sabatino Riontino
- Company Details: Celantur GmbH
- Company Position Title: Celantur GmbH
- Country: Austria
- Linkedin Profile: Yes
- Been thanked: 1 time
Re: Anonymize faces, license plates in E57 photos
Hey, I just found this post and noticed that someone already mentioned Celantur (https://www.celantur.com/) as a possible solution for this problem. As a matter of fact, we offer a solution to automate the anonymization of faces and license plates. We offer two software:
1- Celantur Cloud: user-friendly and pay-per-use option with fast data processing capabilities. Available as a cloud-based SaaS.
2- Celantur Container: Highly scalable Docker container that can be deployed on your local machine, physical servers, or public/private cloud infrastructure. Seamless integration into your data workflows via input and output directories, input and output directories, or RestAPI.
With that being said, we do not support e57 processing at the moment. But we're happy to open a discussion and figure out how that could work. If you're interested, feel free to write me an email at [email protected]
- Mario.
1- Celantur Cloud: user-friendly and pay-per-use option with fast data processing capabilities. Available as a cloud-based SaaS.
2- Celantur Container: Highly scalable Docker container that can be deployed on your local machine, physical servers, or public/private cloud infrastructure. Seamless integration into your data workflows via input and output directories, input and output directories, or RestAPI.
With that being said, we do not support e57 processing at the moment. But we're happy to open a discussion and figure out how that could work. If you're interested, feel free to write me an email at [email protected]
- Mario.