"Can web crawlers revolutionize ecological monitoring?" researchers ask

An interesting article on using the Internet to revolutionize ecological monitoring was published online last spring, and while it received a fair bit of media attention, it passed me by (and I assume many others as well).

The research article was just published in print in the latest copy of Frontiers in Ecology and the Environment, and I think it merits greater attention in the conservation community.

Researchers have proposed a visionary approach that would use automated web crawlers to look at the flow of information across the Internet and detect early warning signals of impending environmental problems before they become catastrophic.

The approach would work as a sort of alert system for particular types of ecological problems that can have quick onset and lead to severe damage - for example, the collapse of a fishery, mass coral bleaching, the invasion of a particularly harmful species like Asian carp entering the Great Lakes, etc.

Typically, conservationists react to these problems after they have already occurred and passed over a threshold of irreversibility. Victor Galaz and fellow researchers contend that the internet offers a revolutionary opportunity to monitor signals of impending ecological problems ahead of time.

Environmental issues like the collapse of a fishery are associated with a complex web of social, economic, and ecological factors that are extremely difficult to track with conventional monitoring methods. However, much of this information is communicated online everyday across countless website.

Web crawlers - the automated programs used by companies like Google to scour the millions of websites across the internet everyday - may be the key to mining this valuable information otherwise hidden in the nooks and crannies of the web.

The researchers give a hypothetical example of using web crawlers to detect signals for a sudden decline in a fishery through observation of  rapid changes in prices, landings, or investments in equipment.

This web-based monitoring approach has already been adopted in the public health sector where it's used to detect early outbreaks of diseases by crawling news wires, websites, local online newspapers, and public health e-mail information services. The study authors note that system "currently captures the first hints of about 40% of the 200–250 outbreaks subsequently investigated and verified by WHO each year."

So the question arises of whether a similar approach could be applied to the field of conservation. For this system to work, we would need to better understand the types of online information that could serve as reliable indicators of ecological decline. This currently sits as a virtually untapped field of inquiry.

Even if we do identify these online indicators, the question arises of how we integrate this information into a coordinated response. This is where the highly decentralized conservation movement faces serious challenges compared to the public health sector.

The authors note, "A relatively strong international organization for human health, with an international mandate – the WHO...is in stark contrast with global environmental governance, which suffers from implementation deficits, serious coordination failures, and inadequate funding."

Clearly, using web crawlers as a coordinated ecological monitoring system is still an idea in its infancy. Nevertheless the approach offers exciting promise about the future of conservation work.

--Reviewed by Rob Goldstein

Galaz, V., Crona, B., Daw, T., Bodin, Ö., Nyström, M., & Olsson, P. (2010). Can web crawlers revolutionize ecological monitoring? Frontiers in Ecology and the Environment, 8 (2), 99-104 DOI: 10.1890/070204