Planning environmental monitoring

When assessing the environmental impact of an event - in our case pesticide treatment of ants or even investigating the effects of the ants on some part of the environment - a few key things need to be considered.

First, the potential non-target species (in the case of pesticides used for ants, this would be other animals - even humans) need to be identified. This is best done through environmental impact assessment, which also involves plans to prevent non-target impacts.

Second, deciding how that monitoring is going to take place. What are you going to measure to determine impacts? Currently in the toolkit we focus on shorter-term assessments of non-target impacts on animals in the environment. We have also added information on monitoring longer-term, broader impacts of the pesticides used - such as chemical residues in soil and water. Currently, there are no studies that we know of that have assessed these potential effects of ant eradication programmes, with the exception of this study on Christmas Island.

Third, when and where the monitoring takes place. This includes planning to be able to separate the effects of the treatment from natural variation in the environment over space and time. This is where BACIPS design comes in. This is a form of experimental design.

What is BACIPS?

BACIPS stands for Before-After Control-Impact Paired Series (BACIPS). To figure out what it means, we can break it down into parts. That way we can understand why the BACIPS design is so good for non-target impact monitoring.

The benefit of a design like BACIPS is that it accounts for sources of variation, so we can be more sure that the results we see are actually due to the work we have done, and not some factors outside our control.

Before-After The key thing you want to find out is that there has been a change after treatment. The change may be positive (target ants have been eradicated or reduced in number). The change may also be negative (non-target effects of treatment) - see the technical note at the foot of the page. But if you only include surveys Before and After treatment and find no difference in ant numbers or non-target impacts, what might the reason be? There might be some sort of seasonal change that happens that by chance occurred while you were doing treatment.

In order to take into account these possible natural variations in time, you need to include a Control-Impact component to the design.

Control-Impact Obviously you need to monitor where the pesticide has been used. This is the Impact site. You want to compare the Impact site to a Control site that is similar, but where the pesticide has not been used.

How similar does the site need to be? Well, it would be best from a scientific point of view to have the Control site as similar to the Impact site as possible - including having the invasive ants. But because the goal is to treat the ants, you might need to choose a Control site that does not have the ants - this is actually the target for what you want to achieve by getting rid of the ants.

But what happens if the Control site and Impact site are very different? It may appear there has been a change as a result of the treatment when there hasn't been or vice versa.

Before-After Control-Impact To cope with the problems of separate Before-After and Control-Impact designs, we combine them together to form a Before-After Control-Impact (BACI) study. This way you can separate the effects of time (Before-After) from the effects of treatment (Control-Impact).

Paired Series

Sometimes impacts might take a while to be seen, or the impacts might only last a short time after the treatment. Having a Series of monitoring events enables you to see these differences. Simultaneous, or Paired, sampling of the control and impact sites at each sampling time point makes it less likely that variation between the two will be caused by differences in timing. Sometimes these designs are also referred to as BACI designs, but repeated over time. Replication If there are multiple sites that need to be treated and monitored (impact sites), be sure to pair each impact site with its own control site, instead of using only one control site. Also, it's a good idea to have more than one measurement or sample at each site to capture within-site variation.

This is known as replication. In some cases, it's not necessary. For example, for the monitoring of non-target effects of pesticides on foxes on the California Channel Islands it was more important to detect effects on individual animals.

However, most studies of non-target effects of ant control programmes often focus on sampling ant and other insects in the area, as these are the most likely to be affected by pesticides.

Replication simply means having more than one sample of something (a study site, pitfall traps etc.). You need replication because there can be a lot of differences between two single samples. This is variation, and can really affect your conclusions. For example, ant nests are not evenly distributed in the environment and placing pitfall traps near to one nest will result in more of those ants than other ants. So, to get a good sample of different ants, you need multiple pitfall traps in an area.

Ideally replication is needed at all 'levels' in your experimental design. For example for pitfall trapping, you need more than one pitfall trap in each transect or plot.You need to know that your results are sound. This is why you have many sites, many transects / plots within sites and many traps within transects / plots. This is 'nested' replication, and removes or minimises the effect of natural variation at different geographic scales (between the sites, between the transects and between the traps). At least two replicates at each level (site, transect / plot, trap) are recommended, but this is sometimes not possible. How to allocate this effort can be complicated, so it's important to keep in mind what is possible in a given place over a given time. Making some effort is better than nothing.

So to put it all together, BACIPS design is simply where you choose a Control site similar to the site that is going to be treated (Impact site), then monitor both sites at the same time, or as close together as possible (Paired), at multiple time points (Series), both Before and After treatment. This type of experimental design is great for distinguishing between differences that are caused by the treatment and those caused by natural variation in the environment over time.

It is important to note that the  BACIPS framework is an ideal to aim for, and other practical considerations - time to monitor, funding, only having one site etc. - will limit what is possible. 

Information sources

Osenberg, Bolker, White, St Mary, Shima. 2006. Statistical issues and study design in ecological restorations : lessons learned from marine reserves. Restoration Ecology in Context. Foundations of restoration ecology. Island Press, Washington, DC, USA

Osenberg, Schmitt, Holbrook, Abu-Saba, Flegal. 1994. Detection of environmental impacts: natural variability, effect size, and power analysis. Ecological Applications 4: 16–30.

Smokorowski, Randall. 2017. Cautions on using the Before-After-Control-Impact design in environmental effects monitoring programs (download 2.2 MB). Facets 2: 212-232

Thault, Kernaléguen, Osenberg, Claudet. 2017. Progressive-Change BACIPS: a flexible approach for environmental impact assessment. Methods in Ecology and Evolution 8:288-296

Technical note

In BACIPS, variables that are measured (e.g., counts of animals, measurements of chemical properties) will have either a positive or negative change over time. Sometimes the variables being measured will increase, but these might reflect negative effects on the environment. For example, if pesticide residues increase, this would be a positive change in the variable, but a negative effect on the environment. In the discussion in this page, for the sake of simplicity, we refer to positive as positive for the environment, and negative as negative for the environment.

Content reviewed by Jeffrey Shima, Victoria University of Wellington, February 2018