Conflicting sphere target distances Faro Scene 2022.1

[lsf]

why are there two (2) "4" spheres?

Great catch, I fixed it, my copy/paste error

[SAttaya]

The summary bothering:"
VXGrid Data Summary.png

The values are displayed in mm. The results at 6.2m distance are better than 2m distance. The red values are really concerning. I really thought that there might be some issue with my S70 or my targets, but this exercise does not confirm this.

What quality/precision do you guys state in your job contracts? 1cm?

go look at the chart above (2nd to last entry on page 5 of this post) from lsf using VXGrid's four (4) files (4th post, page 5).

after checking some of the numbers, I don't understand/can't explain this difference to my own satisfaction
that in two (2) different SCANS that both have the same two (2) targets (assuming the targets haven't moved)
how the distance between the same two targets can be > 4mm different at the ranges of the
targets from the scanner in those scans. using scene 2022.2.0(.10355). Hand fitting the target
or using auto-find/pushpin.

download the scans, look at scans 000 and 005.
look at the targets 1 & 14 in both (lsf's numbering) and targets 1 & 4

it's as if VXGrid's scans 000 and scan 005 were done with different scanners.
(maybe the scanner wasn't warmed up, look at FARO's compensation procedure).

67 (3 x 19) arc seconds at 8 meters, you get 2.6mm (assuming the 19 arc seconds FARO quotes is 1 std dev, 3 std should cover most cases, right?)
the distance between the scanner locations and the target location was typically under 6 meters with that one instance at 8.1m.

This bears some looking into. does it have to do with the horizontal angle from from the scanner to the target? or what.

Another hearty thanks to VXGrid for publishing the data set and
ALSO lsf for not just letting all of us off of the hook

Jcoco3 want to chime in after you run the numbers?

(of course, maybe my math is wrong...it is late on a Friday afternoon

[jedfrechette]

What quality/precision do you guys state in your job contracts? 1cm?

Basically yes.

Keep in mind that all of the manufacturers love to quote their accuracies as 1 standard deviation values. So even a P40 that claims 3 mm 3D accuracy at 50 mm is only going to achieve that accuracy 68% of the time. At a 99% confidence level that scanner's accuracy drops to 9 mm (under test conditions). Under real world conditions, even that accuracy is going to be optimistic as it doesn't account for things like incidence angle, higher or lower reflectivity materials, and various other factors that affect the results.

All that being said, I also wouldn't be surprised if the way Scene fits target spheres is less than optimal. I haven't been following this thread closely so maybe I missed it, but the first thing I would want to do if attempting this type of analysis using spheres is to look at the residuals between the individual points in a scan and the sphere that was fit to those points. Take your point cloud in to Cloudcompare, create a sphere primitive with the size and position of the target that was fit, then generate a heat map showing the residuals between the point cloud and the target sphere. I'd be willing to bet the residuals aren't randomly distributed across the surface of the sphere.

Fitting spheres to lidar scan is actually deceptively difficult. On the one hand you have a flattish surface on the front of the sphere that provides good, high accuracy, returns to the scanner, but that patch doesn't define the sphere very well. As you wrap around the sides of the sphere the scan points provide better definition, but their quality decreases as the incidence angle increases.

[VXGrid]

Alright, here have some data:
4 raw FARO scans (I think it was a S70), with identical spheres

Thank you so much! This data has the same issue. Is it okay if I share some screenshots for the community?

Sure, you can post images, the data set is (not as FARO raw scan data), on our website anyway.

[VXGrid]

[...]


it's as if VXGrid's scans 000 and scan 005 were done with different scanners.
(maybe the scanner wasn't warmed up, look at FARO's compensation procedure).

I can guarantee that I used the identical scanner to create both scans.
In between scan0 to scan5 were scan 1, 2, 3 and 4 of course made.

What do you mean by warming up?
The scanner was acclimated (it was at the location for 12 hours before usage), and it were normal temperatures inside as well as outside.

[VXGrid]

So I was curious, so I did the same in our software and calculated the distances between the spheres.

I compared 33 sphere distances, where
6 fell within 1 mm (18%),
13 fell within 2 mm (39%),
21 fell within 3 mm (63%),
24 fell within 4 mm (72%), and
28 fell within 5 mm (84%).

Biggest differences (over 6 mm) were 7,7 and 6,1 mm.
In scan Inside_000 to Inside_006 and Inside_015 to Inside_000
The distances between scan 000 to scan_006 and scan_015 is around 4.5 and 6.2 meters, so the distances to the scanned spheres from the scanner changed the most with these spheres.

The question is: Which influence is generating these values?
Is it the sphere fitting, or the angular values (depending on where the sphere is located in the scan) or the range finder?

[SAttaya]

[...]


it's as if VXGrid's scans 000 and scan 005 were done with different scanners.
(maybe the scanner wasn't warmed up, look at FARO's compensation procedure).

I can guarantee that I used the identical scanner to create both scans.
In between scan0 to scan5 were scan 1, 2, 3 and 4 of course made.

What do you mean by warming up?
The scanner was acclimated (it was at the location for 12 hours before usage), and it were normal temperatures inside as well as outside.

I didn't mean to imply that different scanners were used.
from the properties recorded by the scanner, they were all the same scanner and approx 70 minutes from scan 000 to 015.
I had checked both the scanner id and the time between scans before I wrote that.

RE the "Warm UP" comment -I seem to remember something from FARO that said the scanner should be warmed up before
do a compensation, but I can not find it now.

back when we used an 120 (DS?) we found that sometimes it made a very slight difference if we warmed
the scanner up such the temp reported by the scanner was approx 40 degree C.
Never ran the test on the S series we use now, nor the Premium.

Thanks again for the sphere distances in your 2nd msg, stated another way:
6 <= 1mm
7 >1<=2 mm
8 >2<=3 mm
3 >3<=4 mm
4 >4<=5 mm
5 >5<=7.7mm

you can use "SELECT" to see which points on the sphere went into
the sphere location calculation routine. Obviously SCENE know to
NOT use some of them, but in some cases the points that in the
the SELECT set are not near what you would it expect.

Anyone from FARO want to comment on:
* how they calculate the center of the sphere and
* how they select the points that go into that algorithm?

[lsf]

So I was curious, so I did the same in our software and calculated the distances between the spheres.

Which software did you use, or you have a custom in-house app that you use for checking things?

I compared 33 sphere distances, where
24 fell within 4 mm (72%), and
28 fell within 5 mm (84%).

Just to make sure that I understand your results correctly, does this mean that 4 (28 - 24) had 5mm mismatch?

[VXGrid]

So I was curious, so I did the same in our software and calculated the distances between the spheres.

Which software did you use, or you have a custom in-house app that you use for checking things?

PointCab Origins, so the sphere estimation is different.

I compared 33 sphere distances, where
24 fell within 4 mm (72%), and
28 fell within 5 mm (84%).

Just to make sure that I understand your results correctly, does this mean that 4 (28 - 24) had 5mm mismatch?

I compared 33 different sphere to sphere distances within a scan, like you did.
28 of the 33 distances differed less than 5 mm.
Like absolute value( (distance sphere A to sphere B in Scan_005) minus (sphere A to sphere B in Scan_006) ).

Since we don't have the real values and only the deviations, this is the best one can see.
Except if we align them and then use the estimated sphere middle points to compare the actual sphere middle points.

[lsf]

Anyone from FARO want to comment on:
* how they calculate the center of the sphere and
* how they select the points that go into that algorithm?

I did reach out to Faro back in December last year before I started this thread (https://laserscanningforum.com/forum/vi ... hp?t=19803). The resolution was to pay $3.9k and send my unit for "Includes complete system inspection, mandatory upgrades and perform a calibration to traceable to international standards for FARO Laser Scanner Focus S 70/150/350. Additional repair charges may apply." As you can see I did not go that route and started with the very basic question: are the 3D distances the same between the same spherical targets across multiple scans.

This one is still on my todo list:

This topic is still on my todo list because I got carried away with math. Let me explain. The core exercise here is to understand why the 3D distances between the same spheres across multiple different scans (individual scans, not registered ones) are not the same. So I went back to the very first step which is matching the spheres in the point cloud. After different experiments it seems that Scene matches the green sphere with some area of the point cloud sphere (why do I think so, because I changed the diameter in the settings, and the green sphere always matches with some points and not with the others), which goes against my intuition. If I was a C++ developer who needed to solve the sphere matching algorithm, I would try to match the sphere between all the points and have a mismatch everywhere instead of aligning the green sphere such that it matches the cloud points only at some part of the surface. With my approach even if the sphere size in the settings is larger/smaller (some range of course), it would still determine the center point correctly and if the green sphere size in the settings is say 1mm smaller than the size in the point cloud, I could see how the cloud points are offset from the green sphere at all surface areas evenly. It might even help in cases when spheres are slightly oval.

So what I got carried away with is I found mathematical algorithm that calculates the radius of the sphere from 4 points of the surface. If the sphere has say 1000 points on the surface, the thought is to take the first 4 points (1 to 4) and calculate the radius, then take points 2-5 etc. So I would end with bunch of radiuses. I would calculate the normal distribution and get the 'bell' curve. My statistical knowledge is a bit rusted but, then I believe I need to take +- 1 standard deviation from the mean and disregard all the other radiuses and outliers. I would then need do calculate another bell curve from this filtered data. I am not sure how many times I would need to do that, but at some point I should get a good 'statistical' radius. I have to think that is how Riscan works?

I've fitted some spheres in Riscan and found a diameter of 0.202m which is 7.9527 inches.