Our imaging system is responsible for finding Joe’s location to within 10 metre accuracy. This must be done quickly and with potentially limited bandwidth.
We know Joe is within 100m of his reported location, and he will be wearing his trusty blue jeans. Our approach is to take an image of the entire area from an altitude of 250 metres. Once Joe is identified we can then using the aircraft’s known position and attitude we can determine his exact location.
The aircraft flies over the target area, its roll and pitch is limited to 0.5 degrees so the aircraft stays level, and captures a series of photos. A coverage map is generated which shows which photo will be the best to download.


Once the best image is chosen this is downloaded via progressive scan jpeg, so that a low resolution pass will appear before the full image is downloaded.


We can then start to look for potential Joe targets. When the image is clicked it does two things. First, it sends the coordinates to the plane and requests a cropped section of the image to be returned so that Joe can be verified in full resolution. Second, it sends those same coordinates to a georeferencing script that calculates Joe’s location.
Joe’s location is also outputted to a KML file so it can be viewed on Google Earth. This way we can cross reference other landmarks to determine the georeferencing calculation was correct. The system diagram below explains the how the different components work in more detail.
A number of sources of error were identified, measured and corrected to achieve 10m
accuracy. These include:
  • Telemetry stream delay – The position and attitude data is captured from the telemetry downlink of the autopilot. Sometimes the data is only sent down every 1s  which means there could be up to 1s of delay between the position where the aircraft actually took the image and where it is recorded. To fix this we sample the position and attitude data every 0.05 seconds and then determine the point before and after the image is taken with the most up to date data, and interpolate between those points. A 0.5 second delay when the plane is travelling at 20m/s can result in a 10m error.
  • Camera offset to autopilot – The camera is attached to the airframe and the autopilot is attached to the airframe. A 1 degree offset of pitch or roll can lead to a 5m error from a height of 250m. To fix this we put the aircraft on a table and put it exactly level as determined by the autopilot. We then dropped a plumb line down from the camera and marked the point in the ground directly below the camera. An image was taken, and if the camera was perfectly aligned with level then the centre of the image should be aligned hit the point marked on the ground. We found that the camera was about 3 degrees out so we just applied an offset in the georeferencing script.
Once Joe’s location has been confirmed, we then determine a suitable landing locat
ion within 30-80m of Joe, and a suitable approach heading for the aircraft to land. We then generate a script that automatically updates the required waypoints on the retrieval aircraft to land in the desired location.