View Full Version : Quantization error

11-25-2005, 12:39 PM
There appears to be a quatization error of about 10 meters in easting when geodetic lat/lon is imported to Mappoint and displayed on a map. This error appears in areas where the absolute value of the longitude is greater than 100, i.e:
longitude = -122.098280, but NOT longitude = -74.009730.
I have noticed this at latitudes = 37.39428.

Steps to reproduce:
1) Create a latitude and longitude data series in Excel where each varies by 0.0001 degree steps. I used 100 steps.
2) Use the data import wizard to import this data to Mappoint.

There is a noticable stair step pattern in locations where there are three integer digits of longitude. The magnitude of the steps is approximately 10 meters, but only in easting, northing is unnaffected.

We would like to show better precision than this on our maps. Is there any way to overcome this limitation?

Eric Frost
11-26-2005, 10:44 PM
I doubt it.. good finding though. Post to the Wish List thread for the next version.

11-29-2005, 02:56 PM
I use Mappoint to plot a tracking report from vehicles, I have the same problem which is really annoying. When you plot the vehicle is knowhere near the roads sometimes and you have to guess what road it is on if the area is built up. I did ask MS about the error and got no response. It would be nice if they rectified the problem.

11-30-2005, 07:15 AM

There is a noticable stair step pattern in locations where there are three integer digits of longitude.

I tryed in code what you explain but do not see something wrong. However I have no roads at a longitude +-100 and mappoint does not zoom in mutch on places where is no road data.

But I dont quite understeand the error you discribe. Can you show it so that I understeand ?

11-30-2005, 07:27 AM
A 10m quantisation error should not intefere with road identification - except possibly at a junction between minor roads, where all bets are off anyway.
Most roads are wider than 10m.

Also, the errors on consumer GPS units often exceed 10m. It varies of course according to reception, antenna location, etc. Also, the consumer GPSs are doing better at extrapolating movement although this often goes wrong - eg. if you change direction or speed.


11-30-2005, 09:02 AM

Richard is right. Also I've seen many companies building the GPS antenna into the inner mirror of vehicles which is cheaper for installation, but then the antenna only see part of sky resulting in higher position dillution. The GPS receiver seeks the best satellites to calc his position, and this is not the same as the satellites with strongest signal.

Setting the HDOP of the receiver improves accuratie, but has disadvantage that only accurate positions are calculated thus resulting in less positions.