Previous material: HWC Tech Challenge Winners Announced, Introduction, Update 1, Update 2

As a quick recap, we chose a ULIS Micro80 Gen2 shutterless microbolometer sensor with 80x80 pixel resolution for work in the Arctic, with polar bears, or any climate below -10°C, and a FLIR Lepton 2.5 with 80 x 60 pixel resolution for work in India with elephants, or any climate above -10°C. After reviewing various thermopile and microbolometer thermal sensors, we chose these two microbolometers because we believe they will give us the necessary image quality for a computer to identify a target animal at the lowest cost. These two sensors have similar capabilities and differ mostly in presence of shutter. A shutter is essential to removing noise from final images taken with a microbolometer and producing a clean image. However, mechanical shutters will freeze in temperatures below -10°C, making a shutterless sensor necessary for work in the Arctic. We will go into the implications of using shutterless sensors in more detail later but it’s important to know that removing noise and producing clean images is more difficult with a shutterless sensor.

Arctic camera prototype

Our arctic camera prototype consisted of:

During our arctic trial, we wanted to investigate the image quality Micro80 Gen2 sensor in cold temperatures and specifically the amount of noise in the final images. We also wanted to test our ability to remove noise from the images using a shutterless correction method. Removing noise from the image is critical to have a clear enough image for a computer algorithm to identify a polar bear as a polar bear.

Additionally, it was equally important to test the power consumption and the PIR sensor trigger. In our prototype camera, the Micro80 Gen2 was always on, but no images would be saved unless the PIR sensors sensed movement. Getting false triggers was a concern, as sometimes changing temperature conditions are enough to mimic the change in IR radiation caused by something warm moving past the sensor. Ittoqqortoormiit, Greenland also experience heavy winds that blow snow through the air which we feared might cause false triggers.

Arctic Testing

We went to Ittoqqortoormiit, Greenland to test our arctic prototype cameras. Polar bears have been visiting the town’s garbage dump, its sanitation workers, and sometimes wandering further into town. We positioned our cameras on the side of buildings near the dump site to monitor activity at the dump and the nearby shoreline, since polar bears come ashore from the sea ice in the adjacent bay and move toward the smell of garbage.

Site challenges

Arctic weather and conditions posed some challenges for installation and maintenance. Polar bear season is during the winter when snow is heavy, temperatures are coldest, and daylight hours are few. High winds after snowfalls can create snow drifts against buildings taller than a person. The cameras had to be placed high enough to avoid snow drifts, but low enough to see an animal walking close to the lens. Short daylight hours and low light angle meant a solar panel could not supply extra power, making the cameras reliant on batteries alone. Cold arctic temperatures reducing battery life meant performing more frequent maintenance or a perhaps purchasing a larger, more costly battery. The low light angle also had to be considered when positioning the cameras, as the thermal sensors cannot be pointed straight at the sun or they become over-saturated and cannot produce images. This is not a problem when the sun is high overhead during the day, but during the winter in the Arctic, the sun stays near the horizon meaning that a camera facing the horizon would put the thermal sensor directly in front of the sun all day.

Shutterless correction for noise removal

All microbolometers have an inherent problem with the sensors responding non-uniformly to incoming radiation. This uneven response creates noise inside the sensor that shows up in any image of a scene, person, or animal taken by the microbolometer. This noise degrades the image, obscuring object or animal and making detection and identification difficult or impossible.

Image corrections are applied to remove this noise from the final image. With shuttered microbolometers, a shutter is closed over the lens, giving the sensor a uniform temperature scene. Any temperature variation detected is, therefore, being generated by the focal plane array as excess noise. This pattern of noise is isolated and removed from subsequent images of people or animals.

With shutterless microbolometers, the image correction must be applied using an algorithm that anticipates the amount of noise in the sensor at a certain temperature. These corrections can be performed in real-time while the image is taken, or after the image has been captured. Real-time algorithms require a lot of up-front calibration of the thermal sensor to isolate noise patterns at different temperatures. They then use the temperature of the sensor at the time of image capture to apply to correct noise removing algorithm. These real-time images are the best at removing noise but take a lot of time and effort to create. Algorithms used in post-processing are already created but must assume the same temperature of the sensor for every image capture. Because the actual temperature of the sensor changes with outside temperature, these algorithms cannot remove all the noise from an image.

We used a post-processing algorithm to remove noise from our arctic images to get a sense of the image quality we could expect without spending months developing a real-time algorithm. This resulted in some noisier images than would be expected if we used a real-time processing algorithm, but we still had some good quality results.

Results

We deployed two prototype cameras at the dump site in Ittoqqortoomiit were we knew we would pick up sanitation workers and hoped to see polar bears. Unfortunately, no polar bears visited the dump site while our cameras were up, so we were unable to see the image quality of a polar bear in arctic conditions. We did have success capturing images of people working at the dump site with good image quality.

The images above, from left to right, show a worker walking in front of the camera, a worker moving hot rubbish burned in the incinerator, and myself standing in front of the camera. There is some noise variation between images, but they have generally good image quality and crisp outlines.

The amount of noise varied between photos depending on the weather conditions and temperature range of objects in the scene. The closer the sensor temperature to that assumed by the post-processing algorithm, the better noise removal in the image. The range of temperatures, or contrast, in the scene also affects noise levels in an image. A scene with low contrast will always make noise more visible than a scene with high contrast.

The images above were taken seconds apart from each other and show me moving in front of the camera. The image on the left is lower contrast (taken before my warm face is in front of the camera) than the image on the right (with my face in front of the camera). You can see more noise in the lower contrast image on the left, showing up as small white spots and stripes, than in the higher contrast image on the right.

Above are two of the noisiest images captured. The left image shows a raven standing on the ground. Its temperature does not contrast very strongly with the snow-covered background. The raven’s small size makes it difficult to identify as a bird, especially with the noise in the image. The image on the right has no obvious subject in the frame and is riddled with stripes and stipples of noise. Low contrast or a different sensor temperature than assumed by the processing algorithm could explain the noise in both images.

We had no obvious false triggers, even on days with heavy wind. The few blank images we had, such as the right image above, were due to a larger field of view on the PIR sensor trigger than the thermal sensor, causing movement outside the thermal sensor's field of view to trigger the PIR sensor. Battery life was a challenge. Not only did the cold temperatures shorten battery life, but the thermal sensor drained more power than it did during testing in the UK. Consequently, the battery lasted only 1/3 of the time predicted, which meant we couldn’t monitor continuously all day and night without charging the battery. This will be an obstacle to overcome for a final camera build.

What's next?

Overall, we’re happy with the results of our first arctic test, but we would still like to test out the Micro80 Gen2 on a polar bear in arctic conditions. We hope to be able to simulate some of these conditions in zoos in cold climates that have polar bears. This will also be a good way to test different shutterless noise removal corrections to have the clearest image possible of a polar bear. The next step is gathering enough images to create a test algorithm that can automatically identify the thermal image of a polar bear and send an alert to the community. We hope we can remove enough noise and get a clear enough image to do this with the Micro80 Gen 2 thermal sensor. However, because of the difficulties of noise removal with shutterless cameras and the low thermal contrast of polar bears, it might be necessary to increase the pixel resolution and thermal sensitivity of the sensor. To test this possibility, we are going to collect some initial images of polar bears using a ULIS Pico160, with a pixel resolution of 160 x 120 and higher thermal sensitivity for more detailed images.

Stay tuned for our next update with results from our field testing in India!

About the Author

Anne Dangerfield is a project manager for the human-wildlife conflict project with the Arribada Initiative, dedicated to affordable, open-source conservation technology solutions.