Anonymize faces, license plates in E57 photos

Please post discussions that do not fit into any other category.
Post Reply
Phips
I have made <0 posts
I have made <0 posts
Posts: 9
Joined: Fri Oct 26, 2018 2:46 pm
5
Full Name: Philipp Mielke
Company Details: STRABAG
Company Position Title: STRABAG
Country: Deutschland

Anonymize faces, license plates in E57 photos

Post by Phips »

Hi all,

more and more companies (especially large ones with own legal departments) are super-sensitive regarding data protection and privacy. At the same time the demand for web viewers to share scan data increases. So we currently face the problem that we somehow have to anonymize recognizable information such as faces and license plates from the generated photos before we can share the data with clients, partners, etc. Obviously, manually checking and editing the point cloud (or photos) of hundreds of scan positions is not an option.

Does anyone have an idea how this could be solved automatically? Any services that can anonymize E57 files? Any workflows?
User avatar
smacl
Global Moderator
Global Moderator
Posts: 1409
Joined: Tue Jan 25, 2011 5:12 pm
13
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 627 times
Been thanked: 657 times
Contact:

Re: Anonymize faces, license plates in E57 photos

Post by smacl »

Phips wrote: Wed Feb 10, 2021 3:06 pm Hi all,

more and more companies (especially large ones with own legal departments) are super-sensitive regarding data protection and privacy. At the same time the demand for web viewers to share scan data increases. So we currently face the problem that we somehow have to anonymize recognizable information such as faces and license plates from the generated photos before we can share the data with clients, partners, etc. Obviously, manually checking and editing the point cloud (or photos) of hundreds of scan positions is not an option.

Does anyone have an idea how this could be solved automatically? Any services that can anonymize E57 files? Any workflows?
Hi Philipp,

At a guess, you would have to anonymize the photography prior to colourizing the point cloud, or anonymize and then re-colour (and also mask intensity for certain areas). There are a number of applications and services to do this for the photophraphy, the trick is transferring this back to the point cloud. As a minimum you need a routine to accurately transform from scan coordinates into picture coordinates for checking purposes.
I've done this in the past for Pegasus 2 but not had much luck with Pegasus ultimate. You may also need an anonymize that will work with spherical photgraphy. The workflow comes down to the equipment and software that you're using,

Shane
Shane MacLaughlin
Atlas Computers Ltd
www.atlascomputers.ie

SCC Point Cloud module
User avatar
Leandre Robitaille
V.I.P Member
V.I.P Member
Posts: 542
Joined: Sat Aug 03, 2019 1:53 am
4
Full Name: Leandre Robitaille
Company Details: Cima+
Company Position Title: Civil Technician - Surveyor
Country: Canada
Has thanked: 61 times
Been thanked: 246 times

Re: Anonymize faces, license plates in E57 photos

Post by Leandre Robitaille »

I have never done this but this is how I would approach the issue. First I would import that e57 in scene 5.5 to conserve the 360 images. Then right click each scans, Operations, color/picture, export color overlay. You would then have all the 360 images in a folder. I would then learn how to use Anonymizer, an AI powered software I know nothing about, but seams to work for that exact purpose.
https://forum.mapillary.com/t/anonymize ... pload/4230 (in this forum they say they corrected 360 images with this software)
https://github.com/understand-ai/anonymizer
Then this is the manual labor again, you would have to manually replace all the 360 panoramic for all the scans. Rigth click, Operations, color/picture, replace color overlay.

This might not be the fastest way, someone might know a faster way to extract/replace the 360images (maybe in cloudcompare? never tried), but hopefully the anonymizer software is a step foward in the right direction for what you want to do.

Good luck ! (keep us updated)
Phips
I have made <0 posts
I have made <0 posts
Posts: 9
Joined: Fri Oct 26, 2018 2:46 pm
5
Full Name: Philipp Mielke
Company Details: STRABAG
Company Position Title: STRABAG
Country: Deutschland

Re: Anonymize faces, license plates in E57 photos

Post by Phips »

Anonymizing the photos is not a problem, there are services that offer this (e.g. https://www.celantur.com/).
The big problem seems to be finding an automated workflow that extracts all the photos from the E57 and writes them back in once they have been blurred. Doing it manually is not really an option. Maybe the scanner-manufacturers have a solution on how to access the data...
User avatar
smacl
Global Moderator
Global Moderator
Posts: 1409
Joined: Tue Jan 25, 2011 5:12 pm
13
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 627 times
Been thanked: 657 times
Contact:

Re: Anonymize faces, license plates in E57 photos

Post by smacl »

Phips wrote: Thu Feb 11, 2021 7:11 am Anonymizing the photos is not a problem, there are services that offer this (e.g. https://www.celantur.com/).
The big problem seems to be finding an automated workflow that extracts all the photos from the E57 and writes them back in once they have been blurred. Doing it manually is not really an option. Maybe the scanner-manufacturers have a solution on how to access the data...
Hi Philipp,

Do you just need to replace the photos with a matching set or are you also looking to update the corresponding scan attributes (e.g. RGB and intensity) based on the changes? Faces are unlikely to be an issue in th scan data but number plates could well be readable from intensity for high density scans. If it is just the former, it should be straightforward enough to develop a solution, either in-house if you have developers available or using a 3rd party, using the E57 SDK. This is something we could do for you if you're looking for options. Re-collouring the scan and masking intensities for blurred areas is more involved and depends on having the correspondance information and algorithms to match scan points to photography. While this might be present in the E57 data the chances are that it is either not there or not suitable, particularly for mobile data. From memory, I think Leica Map factory may offer this as an option, but you'd need to talk to them. Antonio Mendes would be a could guy to contact. Not sure about other manufacturers,

Shane
Shane MacLaughlin
Atlas Computers Ltd
www.atlascomputers.ie

SCC Point Cloud module
cesaratleica
I have made <0 posts
I have made <0 posts
Posts: 9
Joined: Mon Sep 30, 2019 4:10 pm
4
Full Name: Cesar Almeida
Company Details: Leica Geosystems
Company Position Title: Reality Capture Specialist
Country: UK
Linkedin Profile: Yes
Has thanked: 1 time
Been thanked: 1 time

Re: Anonymize faces, license plates in E57 photos

Post by cesaratleica »

smacl wrote: Thu Feb 11, 2021 8:27 am
Phips wrote: Thu Feb 11, 2021 7:11 am Anonymizing the photos is not a problem, there are services that offer this (e.g. https://www.celantur.com/).
The big problem seems to be finding an automated workflow that extracts all the photos from the E57 and writes them back in once they have been blurred. Doing it manually is not really an option. Maybe the scanner-manufacturers have a solution on how to access the data...
Hi Philipp,

Do you just need to replace the photos with a matching set or are you also looking to update the corresponding scan attributes (e.g. RGB and intensity) based on the changes? Faces are unlikely to be an issue in th scan data but number plates could well be readable from intensity for high density scans. If it is just the former, it should be straightforward enough to develop a solution, either in-house if you have developers available or using a 3rd party, using the E57 SDK. This is something we could do for you if you're looking for options. Re-collouring the scan and masking intensities for blurred areas is more involved and depends on having the correspondance information and algorithms to match scan points to photography. While this might be present in the E57 data the chances are that it is either not there or not suitable, particularly for mobile data. From memory, I think Leica Map factory may offer this as an option, but you'd need to talk to them. Antonio Mendes would be a could guy to contact. Not sure about other manufacturers,

Shane
Hi Shane. You are correct. From a development point of view, it's not as simple as just blurring images and importing them back. We have to think about the position, orientation and potential distortion of the images, as the metadata that needs to keep it's integrity over the process of blurring or masking. Then there has to be a process to automatically blur them, and then apply them to the colorization of the point cloud. Accurately and reliably.
So this is something that the manufacturers should be looking at more closely. Sensor integration is the key. How different sensors work together across the whole workflow.

From a Leica perspective, we currently support the blurring of images on the Pegasus 2 This is an machine learning development that we have put in place in the last couple of years. This will blur peoples faces and license plates automatically from the imagery of the Pegasus.
Phips
I have made <0 posts
I have made <0 posts
Posts: 9
Joined: Fri Oct 26, 2018 2:46 pm
5
Full Name: Philipp Mielke
Company Details: STRABAG
Company Position Title: STRABAG
Country: Deutschland

Re: Anonymize faces, license plates in E57 photos

Post by Phips »

I think blurring the panorama images would be fine for now.
While blurring the point cloud would be nice, it's probably more of a long-term objective that should indeed be taken care of by manufacturers. Especially, when thinking of intensity or geometrical information...

Anyway, I've noticed that I'm not familiar how a standard panorama image from a laserscan is generated. Is it a) stitched directly from the captured photos or is it b) a projection of the colorized point cloud? I found contrasting claims.

If it's a), could it be an option to blur the photos before they are used for colorization and panorama generation? As a side effect, the point cloud would be anonymized, too. At least the RGB part.
If it's b), could the generated panorama be extracted, blurred and replaced?
cesaratleica
I have made <0 posts
I have made <0 posts
Posts: 9
Joined: Mon Sep 30, 2019 4:10 pm
4
Full Name: Cesar Almeida
Company Details: Leica Geosystems
Company Position Title: Reality Capture Specialist
Country: UK
Linkedin Profile: Yes
Has thanked: 1 time
Been thanked: 1 time

Re: Anonymize faces, license plates in E57 photos

Post by cesaratleica »

Phips wrote: Thu Feb 11, 2021 10:43 am I think blurring the panorama images would be fine for now.
While blurring the point cloud would be nice, it's probably more of a long-term objective that should indeed be taken care of by manufacturers. Especially, when thinking of intensity or geometrical information...

Anyway, I've noticed that I'm not familiar how a standard panorama image from a laserscan is generated. Is it a) stitched directly from the captured photos or is it b) a projection of the colorized point cloud? I found contrasting claims.

If it's a), could it be an option to blur the photos before they are used for colorization and panorama generation? As a side effect, the point cloud would be anonymized, too. At least the RGB part.
If it's b), could the generated panorama be extracted, blurred and replaced?
Hi.
A panorama image (forget about scanning for a moment), is essencially generated by stiching images together. Go around in 360 degrees, with overlap, and then, depending on the software, stich them together using common points (matching the same pixels in areas of overlap), or automatically. Different softwares will offer some filtering, blending options to improve\remove the blending\stiching effect and contrast\exposure balance.

for your b) scenario, I think it's doable, but I don't see a real application for it. I mean, if a point cloud is colorized, there has to be images to colorize it, so to take that information from the colorized point cloud just seems redundant. So probably possible, arguably not the best course of action.

For scenario a), it's possible for many years in the terrestrial scanning world to generate panoramic images from cameras, edit them(however you want to edit them, including blurring if the photo editing software allows it), and them match them to a point cloud in the registration software. Leica support this workflow with the Cyclone register product.
User avatar
smacl
Global Moderator
Global Moderator
Posts: 1409
Joined: Tue Jan 25, 2011 5:12 pm
13
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 627 times
Been thanked: 657 times
Contact:

Re: Anonymize faces, license plates in E57 photos

Post by smacl »

Phips wrote: Thu Feb 11, 2021 10:43 amAnyway, I've noticed that I'm not familiar how a standard panorama image from a laserscan is generated. Is it a) stitched directly from the captured photos or is it b) a projection of the colorized point cloud? I found contrasting claims.

If it's a), could it be an option to blur the photos before they are used for colorization and panorama generation? As a side effect, the point cloud would be anonymized, too. At least the RGB part.
If it's b), could the generated panorama be extracted, blurred and replaced?
The point cloud colours are always derived from the photography but what format that photography will be in can vary based on the equipment vendor, the flat unstitched photos may not be available and re-stitching might not be available. If you're outsourcing the blurring it would be worth asking whether they can support spherical or panoramic photography.
Shane MacLaughlin
Atlas Computers Ltd
www.atlascomputers.ie

SCC Point Cloud module
mario.riontino
I have made <0 posts
I have made <0 posts
Posts: 5
Joined: Tue May 04, 2021 12:43 pm
2
Full Name: Mario Sabatino Riontino
Company Details: Celantur GmbH
Company Position Title: Celantur GmbH
Country: Austria
Linkedin Profile: Yes
Been thanked: 1 time

Re: Anonymize faces, license plates in E57 photos

Post by mario.riontino »

Hey, I just found this post and noticed that someone already mentioned Celantur (https://www.celantur.com/) as a possible solution for this problem. As a matter of fact, we offer a solution to automate the anonymization of faces and license plates. We offer two software:

1- Celantur Cloud: user-friendly and pay-per-use option with fast data processing capabilities. Available as a cloud-based SaaS.
2- Celantur Container: Highly scalable Docker container that can be deployed on your local machine, physical servers, or public/private cloud infrastructure. Seamless integration into your data workflows via input and output directories, input and output directories, or RestAPI.

With that being said, we do not support e57 processing at the moment. But we're happy to open a discussion and figure out how that could work. If you're interested, feel free to write me an email at [email protected]

- Mario.
Post Reply

Return to “Any Other Issues”