article / 17 February 2020

HWC Tech Challenge Update: Thermal Elephant Alert System

The Arribada Initiative is back with an update on their thermal elephant alert system which aims to reduce human-elephant conflict (HEC). The success of their system rests on the ability of a camera to accurately distinguish an elephant from other subjects that pass by. And the success of machine learning algorithms often rests on the quality of their training datasets. In this update, the team has been at ZSL Whipsnade Zoo building an extensive thermal image library of elephants and other subjects. 

Our idea for an alert system combines low-cost thermal sensors, machine learning, and wireless transmission to build an affordable system that alerts communities to elephant presence, giving them a chance to safely act before an HEC incident occurs. We decided on thermal sensors because they perform better than optical camera traps at night and in low light conditions.

Combining low-cost, low-resolution thermal sensors and machine learning for automatic ID are unique to our system.  Low-resolution sensors balance the necessary resolution for detection with an affordable price. Automatic alerts of elephant presence warn residents of potentially dangerous situations, help create reliable reports and understand elephant movement to help develop long-term conflict management strategies.

Project phases

  1. Identify the best, affordable thermal sensor for our application through research and testing at controlled elephant habitats and field sites.
  2. Prove that images taken with an affordable, low-resolution thermal sensor are suitable for machine learning by taking thousands of thermal elephant and human photos, creating a labelled database, and creating a machine learning algorithm.
  3. Design and build the first field-ready prototype of the thermal camera with power efficiency, solar power, automatic identification and alert transmission over cellular and/or long-range radio (LoRa) networks.

Tracks left by crop-raiding elephants in rice paddy field in Assam, India. Photo by Anne Dangerfield

Phase 1 of our project was completed in 2018 by trialing several different thermal sensors at ZSL Whipsnade Zoo’s elephant habitat and several field sites in Assam, India. Reports on our progress during stage 1 can be found on WILDLABS by searching for the Arribada Initiative (most recent update here). This initial testing lead to us choosing to use the FLIR Lepton series thermal sensors for our system.

elephant thermal image 1 elephant thermal image 2

Thermal elephant images taken at a HEC site in Assam, India

Phase 2 - Building a database of thermal elephants

The success of our thermal elephant alert system rests on the ability of a camera to accurately distinguish an elephant from other subjects that pass by. And the success of machine learning algorithms often rests on the quality of their training datasets. In order to prove a computer could recognize the low-resolution silhouette of an elephant, we needed to create an extensive thermal image library of elephants and other subjects.

We returned to ZSL Whipsnade Zoo’s elephant habitat to build our image library. While we did manage to photograph wild elephants on our FLIR Lepton 2.5 thermal cameras during field testing in Assam, India, the elephant habitat at ZSL provided constant opportunities to photograph elephants in a variety of conditions. The zoo habitat was also a good place to gather images with both the FLIR Lepton 2.5 80x60 resolution sensor and the FLIR Lepton 3.5 160x120 resolution sensor. While the FLIR Lepton 2.5 sensor proved to take recognizable images in the field, we wanted to directly compare the performance of computer recognition between the two sensors, especially with smaller or more distant subjects.

Thermal cameras set up and taking pictures at ZSL Whipsnade Zoo’s elephant habitat. Photo by Sophie Vines, courtesy of ZSL.

Data collection methodology

Our data collection intern, Sophie Vines, took pictures of elephants at all different angles and distances, as well as in different environmental conditions, in groups and standing alone.  This created a fully comprehensive library of images to train the algorithm to recognize elephants in many different scenarios. We primarily used the same raspberry pi-based automatic cameras used in India to collected data at the zoo. Because these cameras’ motion detectors only trigger with subjects less than 12 meters, we decided to use the camera’s time-lapse mode, programmed to take images every 5 seconds. This allowed us to capture elephants at any distance from the camera.

This is how Sophie describes her data collection experience:

I set up the automatic cameras on tripods on the edge of the elephants’ outside grass paddock. Because I learned their routine, I would be ready for them at the paddock entrance at 10am when they ambled out for the day and around 4pm when they would troop back inside. Close-up shots were almost guaranteed as they had to walk right past the cameras on their way in or out of the barn. During the day, I could get many shots of the elephants at different distances out in the paddock. There were also spontaneous moments where one or more of the female elephants would come up to the camera, sometimes to get a drink from the nearby water trough or to suss me and my equipment out.

To maximize data collection, cameras were set up in the elephant barn to capture images overnight. With the automatic cameras set to capture images at 5 second intervals, I managed to gather around 5000 images of elephants under 10m from the camera in 2 weeks. Leaving the camera indoors also meant I could capture images with different background conditions than the images taken in the outdoor paddock. The cameras were also out while zookeepers worked in the elephant barn to increase the number of human images in the database.

I also recorded different variables that could affect the computer’s ability to accurately recognize a subject and could be used to understand the limitations of the computer identification. I estimated distance to human and elephant subjects by measuring the distance to items in the paddock and comparing them to the location of the subject in the image. I also recorded the temperature at the time of the image capture to establish if the computer had difficulty differentiating objects from the background when temperatures were similar. This became difficult when cameras were left for extended periods when temperature fluctuated. At these times, a temperature probe was left out which recorded maximum, minimum and current temperature to establish a range for the time period.

elephant thermal image 3 elephant thermal image 4 elephant thermal image 5 

Example of thermal human and elephant images taken at Whipsnade Zoo​

Image labeling

We teamed up with Liverpool John Moores University who are building the computer recognition algorithm. To best train the computer, our intern also annotated the images with boxes outlining the subjects and labels describing what is in the box, human or elephant. We labeled all our images using DeebLabel, a program built by the university team for image annotation.

Results

  • Number of photos collected: 36,643
  • Number of photos labelled: 28,048
  • Number of hours it took to take all images: ~160 hours, approx. 10hrs per week over 4 months
  • Number of hours it took to label: ~112 hours, approx. 7 hours per week over 4 months

Going forward

We are currently using this database to train a computer to recognize humans and elephants. We have only preliminary results at this time, but we hope to share the results of the full algorithm work once it is finished.

Thanks to WWF Netherlands and WILDLABS who funded this project in 2018 as a winner of the Human-Wildlife Conflict Tech Challenge.


Add the first post in this thread.

Want to share your own conservation tech experiences and expertise with our growing global community? Login or register to start posting!