From the Field: Developing a new camera trap data management tool

Frustrated by the limitations of the tools that were available for managing large camera trap data sets, Heidi Hendry and Chris Mann set out to develop something that met their needs, and thus, Camelot was born. 

In this From the Field interview, Heidi and Chris share the evolution of Camelot. the open-source, web-based tool they developed to help wildlife conservationists with camera trapping. Crucially, it does not do sophisticated data analysis itself; it simply manages camera trap data. Instead of having it do all things, they've taken the apporach of having it do that job really, really well and offer the integration points so that an organisation using it can have it work with whichever ecosystem of technologies best suit their needs. Read on to find out more about the tool. 

Date published: 2017/07/07

Q: How did you first get the idea to use this technology for your work?

Heidi: I was working for Fauna & Flora International in Hanoi Vietnam as the IT Manager, and the Country Director, Dr Ben Rawson, asked if I could do some data work as well. I was given 40GB of photos and asked to “do something with them”. The original aim of the survey was to discover whether the Clouded Leopard was in the forest in Northern Vietnam. However, the image collectors had realised that there were none, and so motivation to analyse the images had lapsed. Ben suggested the challenge of analysing general species abundance in the forest. Ben pointed me in the direction of what others do with image files, which was to sort them into folders based on the trap station, the camera, the species and then the number of animals in them. If there were 2 species in an image, then the photo was copied and stored in 2 different places. I had an instant dislike of doing any kind of analysis where the key part was creating folders and dragging files around, and I was sure that there had to be a better way to do it. I went hunting for other camera trap software, but though I tried many different applications, they were not well written, too complex, needed other proprietary software or just didn’t run.

So I started talking to Chris about it.

Chris: The data we were working with was spread across a couple of Excel workbooks, each with a number of worksheets, and with a basic data hierarchy created using folder names. In an ideal world, we would then turn to technologies such as CamtrapR or the tools published by the Smallcats Conservation Alliance to analyse this.

However, the problem with collecting data into spreadsheets and named folders is that there’s not much in the way of validation -- even if there is one data-entry mistake in every thousand entries, that is still a significant number of errors in the final data.

So first and foremost for us was finding a solution which would help us verify and clean this data. Not finding any existing technology to do this, Camelot was born: as a tool which would scan this data to identify inconsistencies. It has obviously grown a bit from these humble beginnings, but that’s how it all started.

Camelot interface 1

Q: What challenge has this technology helped you overcome?

Heidi: The big issue for me was the number of errors in the data that I received. I had received 40GB of photos, one spreadsheet and a map. The spreadsheet had not been completed correctly, image files had been stored in the wrong folders, images had the wrong date and time and the original researchers had left the organisation. I started with the methodologies that were documented in the scientific literature, but there were some assumptions made with those methodologies that I could not fulfil. I needed something that assumed that errors would exist and would give me a way to pull those “needles” out of my “haystack”.

Camelot has allowed us to take images and data that were created for one purpose, and re-use it for another.

The other challenge was getting a good visual of the images along with the data for categorising it. I’m quite visual, and I disliked having to switch views between a photo and a text file.

Q: Do you use specific criteria to select the technology or model you use?

Chris: It’s easy to get caught up in making evaluations based on specific objective qualities. The functionality or features of a technology are important, but it’s useful to look beyond these to assess a piece of technology within the broader picture.

There are many subjective qualities of technology: is it reliable, is it fast, is it intuitive, and so on. These can be very tough to measure, and so are often overlooked when assessing specific criteria.

But also it helps to see technology as a piece in a jigsaw. The ideal solution will often require tying together many pieces of technology with processes and individuals’ skills. It’s easy to miss opportunities to identify integration points unless not only considering the what (objective), but also the how (subjective).

With Camelot we try to balance these (often conflicting) qualities. Camelot does not do sophisticated data analysis itself. It just manages your camera trap data. We try to have it do that really, really well and offer the integration points so that an organisation using it can have it work with whichever ecosystem of technologies best suit their needs.

Heidi: I wanted it to be easy to use. I’ve used many pieces of software over my career in I.T. and non-intuitive software which over-uses the mouse is annoying, and time-wasting. We put a lot of effort into making keyboard shortcuts into the software. This increases efficiency as you have to move your hand from keyboard to mouse much less. Finger movements are faster than lower arm movements.

We could see that there was a gap between capturing the photos and the excellent statistical analysis available from CamTrapR, and we honed in on that gap, and tried to make Camelot work really well in there.

Q: What were some of the biggest challenges you faced using this technology in your location?

Heidi: Because we were developing the software as we went, it was a bit difficult to explain to my Vietnamese counterparts how to use the software. At that point we hadn’t developed documentation.

My project has been interrupted due to personnel changes, and so development stalled for a while, as I was the only “end-user”. We were so happy to launch it publicly, because now we have others using it in real-world situations and they are finding bugs and making usability suggestions that we had not found. Currently, our available time is limited to produce training or explain how to use the software. Fortunately, most people who have found our software via the camera trap forums, ResearchGate, Google or other locations seem to have picked it up quickly.

Camelot interface 2

Q: What are some of the shortcomings of the technology you’re using for your work that you’d like to see addressed?

Chris: We have a few open feature requests in Camelot, specifically around support for uploading and managing video files, the ability to add log-ins and user access controls and the ability to customise the types of data gathered when adding sightings.  The #1 barrier to solving these problems is limitations on our time.

Camelot is open-source though, and we gladly accept help towards implementing these features or others should anyone else wish to contribute.

Heidi: I would really like to have an easier way for end-users to develop reports. It’s better than what was previously available, but it’s still a little “code”-y for non-programmers.

What we would really like is for a team using the software and needing the additional features to find some funds to hire us. Then we could take some “leave without pay” to develop the necessary features to make this suitable for larger scale projects.

Q: Have there been any unexpected positives for using this technology? What are the most surprising findings that the technology has helped you to discover?

Heidi: We discovered there is a real niche for software development for conservation. And specifically, small scale, desktop style applications, as scientists are keen to keep control of their data. Most of the rest of the IT industry is pushing hard towards cloud applications, as they are always available, but scientists have big concerns about data ownership.

I’ve shown this software to a few people and they thought of other uses we hadn’t even considered. One example was to use it for longitudinal studies in a field lab. Over time it would be possible to build up multiple studies in the software, and then run reports cross-study. I was invited to present on Camelot at the Vietnam Tech Conference in Ho Chi Minh City, and we are now having international schools reaching out to us about how to use camera traps.

Also, there is a group in Northern Queensland laying camera traps with the hope of finding an animal that is reportedly similar to the extinct Tasmanian Tiger I reached out to them and told them about Camelot, and they were very interested in it. So, who knows, our software could be involved in one of the most amazing discoveries of our century!

Q: What advice would you give other groups such as yours that might be thinking about using this technology in their work?

Heidi: Use it! Camelot is the best camera trap software available and we are working on it all the time.

Are you interested in using Camelot in your work? Keen to help develop the software further? Heidi and Chris are hosting a conversation about the tool over in our Camera Trap group, so join the discussion if you have any questions or feedback. 

About the Authors

Heidi is more than her job, though she currently works at British International School, Hanoi as IT Manager and Innovator. Heidi’s husband and 2 children are kind enough to free her up to volunteer for Fauna & Flora International and complete a Masters in Information Technology (Data Analytics) on the side.

Chris is a Software Developer at Atlassian with a passion for wildlife conservation. Chris’ natural habitat is at a keyboard, building software to help those making the world a better place achieve that little bit more. When not turning coffee into new technology, he spends his time reading, and enjoying the nature around Sydney, Australia.