The Melbgarten is a branch of the University Botanical Gardens and is located on the Venusberg in Bonn (① in Figure 2). The site partially borders a nature reserve and has a slight slope towards the west (LANUV NRW 2013). At the end of the Second World War, the area of the Melbgarten was donated by the city of Bonn to the university to establish experimental facilities (Botanic Gardens Bonn 2023). Completely enclosed by a fence, with power supply and internet access, the garden area is well suited for experiments with sensitive electronic equipment. The site was used to set up a fully equipped prototype of AMMOD (Figure 3). Due to budgetary constraints, the Melgarten site remained the only site where all modules were operated simultaneously. Thus, the stationary GC-IMS device (Smellscapes module) was not set up at the other sites, but visits with the mobile version of the device were made to the other locations.

Figure 3.

The Melbgarten site in November 2020, looking west. The control cabinet of the base station is approximately in the centre of the picture, to the left of it the automated insect trap (but without the Malaise net) and its solar panels can be seen. – Photograph: G. F. Tschan.

For this test phase of the AMMOD design, an external power supply was planned, as the actual consumption of electricity first had to be assessed before a completely autonomous supply would be possible. However, after an initial site visit in December 2019, the first component to be permanently installed in July 2020 was an automated Malaise trap (Barcoding module) that fully autonomously collects insect samples with the help of two solar panels (Figure 4). Power and internet cables specifically for the project were laid later that year, and could be used from within the control cabinet from November 2020.

Figure 4.

The automated malaise trap of the Metabarcoding module, July 2020. Note the two solar panels, each facing in a different direction. The garden’s greenhouse can be seen in the background on the right. – Photograph: G. F. Tschan.

Between 2020 and 2022, all sensors were successively installed at the Melgarten site (Malaise and pollen trap, Smellscapes module, wildlife cameras and animal sound recorders). Many of the components received updates during the course of the project, especially the acoustics and visual monitoring devices, as well as the base station. Some devices such as the pollen trap were still in operation in 2023 (Figure 5).

Figure 5.

The automated pollen trap of the Metabarcoding module, February 2022. In the beginning, and at the end of the sampling season, the external power supply was used, when the sunlight was not strong enough to enable adequate supply via the solar panels. – Photograph: G. F. Tschan.

An important contribution to the understanding and subsequent acceptance of the use of the AMMOD technologies remained outreach, more specifically the explanation of the devices used on-site to interested visitors. Since the Melbgarten site is not open to the public, specific events were used to promote the project locally in 2022 and 2023, when the collections can be visited during the ‘Spring Festival’ (cf. Botanic Gardens Bonn 2023). In addition, some of the AMMOD equipment was presented to the public during the University of Bonn’s ‘Biodiversity Day’ in the same years. The AMMOD project and the Melbgarten site were also featured on German public-service television in February 2023 (ZDF 2023).

The ecological research station Britz of the Thünen Institute for Forest Ecosystems (Thünen 2023) was used as the second AMMOD experimental site (② in Figure 2). The site near Eberswalde covers an area of 4 hectares, is surrounded by pine forest and comprises different forest structures and open areas (Figure 6). It is a Level II plot of the nationwide forest monitoring program. Through a cooperation agreement between the Museum für Naturkunde Berlin and the Thünen Institute, use of the site for the AMMOD project was made possible. The Thünen Institute also contributed relevant data from the forest monitoring to the AMMOD project. The research station has an internet connection on site (both LAN and WLAN), which could be used for the project. In addition, the site is completely enclosed by a fence and Thünen Institute staff are on site during the week.

Figure 6.

Habitat structures at the ecological research station Britz. (A) Coniferous forest. (B) Open space. – Photographs: K.-H. Frommolt.

Due to the relative closeness of the Museum für Naturkunde Berlin, regular visits of project personnel could be made to the site. In contrast to the Melbgarten site, however, not all devices were permanently employed here. As a benefit, the bioacoustic sensors received special attention at Britz (Figure 7).

Figure 7.

Autonomous acoustic recording unit at Britz. The sounds in the audible range are continuously recorded in loops of 5 minutes recording and 10 minutes pause. – Photograph: K.-H. Frommolt.

Through the Museum of Nature Hamburg (formerly CeNak), contact was established with the operators of the so-called ‘Energieberg’ (literally meaning ‘energy mountain’) in Georgswerder, which is located northeast of Hamburg’s Wilhelmsburg quarter (Hamburg 2023). The Energieberg is a former landfill site that was covered with trees and undergrowth and had already been used by the Museum of Nature for setting up Malaise traps (③ in Figure 2). The site provides an area enclosed by a fence where a base station could be placed and connected to the electricity grid. Its central location offered a convenient infrastructure to test the first prototypes. Accessibility to the Energieberg, mobile phone reception and power supply could be ensured.

The industrial environment of the Energieberg, on the other hand, posed a challenge for the sensors. In particular, the acoustic sensors could only be used to a limited extent due to the noise pollution and were also only used for short time periods. The site was not commissioned for the project until the first experiences and results from the Melbgarten site were available, which was at the end of 2021. A base station and some of the sensors were then set up. The site was used for the project until May 2023.

The volatile organic compounds (VOC) released by living organisms carry information about their identity and physiological status. Kesselmeier and Staudt (1999) gave a broad overview of various VOC patterns characteristic for different plant species and the influence of light and temperature on their expression. The emissions of VOC are influenced by various abiotic and biotic factors, such as seasonal variation (Bracho-Nunez et al. 2013; Bourtsoukidis et al. 2014; Brilli et al. 2016), air pollution (Li et al. 2017), weather (Šimpraga et al. 2011; Niinemets and Sun 2014; Llusia et al. 2015) and even herbivore activities (Clavijo McCormick et al. 2012; Erb et al. 2015; Bouwmeester et al. 2019).

VOC emissions from vegetation (plant volatile compounds = pVOC) are primarily determined by the taxonomic composition, the relative abundance of individual taxa and their phenology (Kigathi et al. 2019). Taxonomic composition and abundance of taxa change slowly and minor changes may be difficult to trace, especially for rare taxa. However, major components of the vegetation and especially their phenology (bud break, flowering, fruiting, leaf fall, attack of pest insects, effects of pesticides, ploughing) should leave a strong imprint on VOC patterns. Highly resolved VOC-patterns would thus provide a uniquely detailed picture of the reaction of plant communities to short term weather patterns on an hourly rather than daily or weekly basis (as typical of phenological calendars). These data could be invaluable for understanding and interpolating ecosystem reactions in view of the complex and poorly understood reactions of different plant species to changing weather patterns in times of global climate change (Peñuelas und Llusià 2003; Yuan et al. 2009).

Plant abundance and phenology are of crucial importance for the consumers in a habitat and have a dramatic influence on the entire food chain. Thus, detailed real-time data on vegetation development would permit conclusions on ecosystem integrity and trends. A simultaneous documentation of anthropogenic compounds released into the ambient air (e. g. pesticides) would be of particular interest for understanding ecosystem reactions to xenobiotics. However, environmental VOC (eVOC; Sum of the VOCs that are in the ambient air at a given point in time) or more precisely plant VOC (pVOC) concentrations in ambient air are in the lower ppb down to the ppt level. The expected mixture of compounds is extremely complex and humid, and could be influenced by biogenic and anthropogenic emissions. Therefore, a sensitive and selective analytical tool is required for a continuous monitoring of characteristic volatiles. The most common and well-established analytical technique is mass spectrometry combined with GC pre-separation or GC combined with flame ionisation detectors (FID). However, broader time resolved measurements of VOC-fluxes using GC-MS or GC-FID often rely on highly sophisticated and extremely costly experimental set-ups, i. e. the Amazonian tall tower observatory or the prophet tower (Yáñez-Serrano et al. 2018; Fischer et al. 2021). Therefore, a rapid and automated analytical tool providing the necessary high selectivity and sensitivity is the ideal alternative.

Ion mobility spectrometry (IMS) is an analytical tool for detecting traces of ionised molecules in the gas phase after separation for size and shape. It is an extremely sensitive method that can detect molecules in the low ppb to ppt concentration range.

In general, a measurement can be divided into two phases, first the ionisation phase and second the separation phase. In the first phase, molecules are ionised in the ionisation chamber. Most commonly this is achieved via a ß-radiation source, which ionizes the drift gas flushing through the spectrometer, thus forming the so-called reactant ions in the ionization region. If a sample with other gas-phase compounds is introduced in ionization region, analyte molecules are ionized mainly by proton or charge transfer. In the second phase, separation of ionised molecules occurs. For this, the ions are accelerated in a weak electric field towards the detector, but only clouds of ions are introduced periodically into the drift region by a Bradbury-Nilsson ion grid. During their drift from the grid to the detector, the ions collide with the molecules of the drift gas counterflow (Figure 1).

Figure 1.

Schematic illustration of an ion mobility spectrometer. Analytes are ionized and then accelerated in an electric field towards the detector. The drift time is influenced by the collision frequency of analytes with an opposing drift gas.

Collision frequency depends on size and shape of the ions, thus influencing drift velocity. Drift velocity can be determined by measuring the drift time of the particular ions under known drift length and the ion mobility is obtained by normalization to electric field strength. Further normalization of the measured drift time to the position of the reactant ion peak (RIP) is leading to reproducible relative ion mobilities, which are independent of instrumentation and environmental parameters such as temperature and air pressure (Vautz et al. 2009).

Such separation and detection can be operated in the positive and negative ion mode just by changing the polarity of the electric field. Thus, alternating analysis of positive and negative ions is carried out. For details of ionisation and separation in ion mobility spectrometry see Eiceman et al. (2016).

Coupling fast gas-chromatographic (GC) pre-separation to IMS provides the characteristic retention time as additional measure for the identification of the analyte and furthermore avoids clustering of different analytes in very complex mixtures in the ionisation and drift region of the IMS. Analysis time of a full GC-IMS run is in the range of 10–30 minutes, depending on the experimental setup of the GC. Additionally, innovative in-line MEMS-based (microelectromechanical systems) in-line pre-concentration systems are applied. This further increases the sensitivity by ca. 1–3 orders of magnitude without increasing the total analysis time and allows automation of pre-concentration and analysis without separating these steps in time and space, enabling a high sample throughput (Liedtke et al. 2019; Vautz et al. 2018).

After construction of the stationary GC-IMS for continuous measurements and optimization for the detection of VOCs in the ambient air, the prototype was implemented for long-term, time resolved measurements in the study area. Furthermore, a mobile version of the GC-IMS with the similar experimental setup was provided for measurements in the lab or in the field, e. g. directly to characterize different plants and plant parts. Concomitantly, a reference database was compiled. This includes an inventory of frequently occurring species in the Melbgarten, as well as a substance database for identification and characteristic emission patterns of common plant species. The aim is to use this reference database to clearly identify plant-related volatiles and, if possible, to assign signals from the long-term measurements to individual plants. In addition, trends are to be analyzed via the time-resolved measurements with regard to seasonality, diurnal and weekly variations and on the influence of particular weather conditions.

The resulting initial questions were “Are there differences in the emission patterns of different plants?”, “Can we detect pVOC signals in the ambient air?”, “Can we assign detected signals to plant sources?”, “Are there substances that show a diurnal/weekly cycle?”, “How do the emission data correlate with abiotic factors (temperature, humidity)?”.

In the following, we want to go into more detail about the methods and the structure of the station, as well as the first results of long-term time resolved measurements.

The sampling site is located at the Melbgarten (Bonn, Germany) at 50.71297°N, 7.09035°E (Figure 2). The Melbgarten is an experimental station of Bonn University Botanic Gardens. The area is surrounded by a forested area (Melbtal) with typical European tree species (e. g. Fagus, Quercus, Acer) and pastures. The nearest major road is located approximately 150 m west of the monitoring station and is shielded by vegetation. Built-up areas are situated some 100 m north and 150 m west of the station. Due to the distance to roads and human settlements, anthropogenic influences on emissions are expected to play a subordinate role. The station is placed on sloping meadow with herbaceous plants and scattered trees. This meadow is mowed once a year (late June). A large part of the area is dominated by Poaceae (Gramineae) and Urtica dioica. In addition, there are various ecosystem-typical representatives of plants from various families, with Asteraceae, Lamiaceae and Fabaceae contributing most to the diversity of the plant community (Appendix 6.1). Sorbus and Sambucus nigra are the typical trees. The meadow borders on a lawn, which is cut at regular intervals (ca. every four weeks).

Figure 2.

Detailed map of the immediate vicinity of the Melbgarten (yellow area) and the localization in the greater Bonn area. Base map: OpenStreetMap (OSM). Additional layers: Geoportal NRW (Naturschutzgebiete, Straßen, Wasserfläche, Bebauung).

Two identical ion mobility spectrometers are used for the measurements in the Smellscapes module. The first device is designated to stationary use and is installed in the monitoring station at the Melbgarten. The second device is additionally equipped with batteries and is used for reference measurements in the laboratory or mobile measurements in target areas.

The ion mobility spectrometers are ppq-tec-GC-IMS from ION-GAS GmbH (Dortmund, Germany) based on hardware provided by STEP GmbH (Pockau-Lengefeld, Germany). The default sample loop was substituted with a MEMS-based in-line pre-concentration chip filled with Carbograph 4 as adsorbent (CNR-IMM, Bologna, Italy, for detailed description see Liedtke et al. 2019) for increased sensitivity. We used a MXT-200 capillary column (30 m × 0.53 mm, 1.5 µm coating) for GC pre-separation, which is operated isothermally at 80 °C, with a carrier gas flow (filtered air from internal gas circuit) of 21 mL min^-1.

The internal gas flow is purified via a filter system. In total, the GC-IMS has two external filters (Molsieve and activated carbon) and four larger internal filters filled with molsieve 9 Å. Filters need to be replaced approx. after 12–18 months. The filters are regenerated externally by the STEP GmbH.

Ionization is carried out with a tritium source of β-radiation (100 MBq). Drift length of the IMS is 5.61 cm. The drift tube is operated isothermally at 70 °C at a field strength of 300 V cm^-1. The instrument provides an automatic polarity switch between measurements, which allows measurements to be made in both positive and negative modes.

The ppq-tec-GC-IMSs have built-in computers that permit on-site control, but also permit remote access via the internet (e. g. via TeamViewer). Data is stored on a built-in 256 GB hard disk. A user manual for detailed technical information and operating instructions of the ppq-tec-GC-IMS is available from the Nees-Institute and ION-GAS GmbH.

To protect the GC-IMS from harmful influences (weather, vandalism etc.), it is housed in a safety cabinet (SciCab12, Figure 3). The sample inlet is located at a height of approx. 2 m outside this cabinet. To prevent contamination of the GC-IMS filters with pollutants such as pollen or other microparticles, the ambient air is initially filtered using a PTFE-round filter (diameter 47 mm) with a pore size of 6 µm and a thickness of 0.1 mm (RCT- SVX from RCT Reichelt Chemietechnik GmbH & Co., Germany). The sample inlet and the GC-IMS are connected through a PFA-tube (diameter ¼ inch). A suction pump regulates an ambient air flow of 500 mL min^-1. The GC-IMS is connected via a T-junction with the constant air flow of ambient air.

Figure 3.

Stationary GC-IMS in the safety cabinet (A, B). the safety cabinet is located on a green area in the Melbgarten (C). The samples are taken at a height of approximately 2 m (D).

Time-resolved measurements of VOCs have been performed at the sampling site in the Melbgarten since 2021-03-22 (including pauses for optimization and maintenance). The main intervals of measurements were from 2021-06-15 to 2021-08-13 and from 2022-03-22 ongoing.

The samples for the GC-IMS are taken through the T-junction at a sampling rate of ~ 60 mL min^-1 from the constant flow of ambient air. For the long-term monitoring, a total sample volume of 1000 mL was used for the enrichment (sampling time approx. 20 minutes, enrichment temperature = 43 °C). Subsequently, substances were released via heat desorption (50 °C for 5 s, then temperature ramp to 290 °C in 5 s, 290 °C for 12 s) and transferred to the GC-IMS for separation and detection. To eliminate carry-over, the pre-concentrator chip is purged at 300 °C for 2 minutes prior to each measurement. Furthermore, the IMS and GC should be baked out at regular intervals to prevent the accumulation of impurities. For this purpose, there is an internal bake-out procedure in which the IMS and GC column are heated above the usual sample temperature for 24 hours.

The duration of the GC pre-separation is 25 min, followed by rapid mobility separation and detection in the IMS (a few µs). After each measurement, the polarity of the IMS switches automatically for the next measurement to cover a wider range of compounds (positive and negative ionisation modes address different substance classes). Combined with a short sampling break (3 min) after each measurement, we are able to record one measurement per hour. In-line enrichment and analysis are fully automated and run 24/7. The data generated, including telemetry data, are directly uploaded into the AMMOD-cloud and are additionally stored on the built-in hard disk.

For signal identification we generated a reference database of common pVOCs and some characteristic anthropogenic compounds. Substances of interest were selected based on literature research and GC/MS reference measurements. Measurements of pure reference substance were conducted for validation. Furthermore, some substances could be identified by comparing Kovats retention indices against a series of n-ketones combined with GC/MS refe­rence measurements. However, this is only suitable for less complex mixtures of volatile compounds.

The database contains signals for 26 aliphatic hydrocarbons, 27 aromatic compounds, 27 Monoterpenes and two nitrogen containing compounds. Most of these substances are biogenic, but they also include some predominantly anthropogenic substances such as toluene or xylenes. A list of all the substances identified is provided in Appendix 6.2 (Table A2).

Reference measurements of individual plants were carried out in the laboratory and in the field. For measurements in the field, a means of transport (pram) was converted as required and equipped with additional storage area, suspension and fixing options (Figure 4). An additional battery pack makes autonomous operation for up to eight hours possible.

Figure 4.

Exemplary images for measurements of flower volatiles in the genus Narcissus in the field (A) and in the laboratory (B).

The sample inlet is positioned directly in front of the relevant plant part (e. g. the flower) and an appropriate sample volume (between 10 and 100 mL, depending on pVOC concentration) is taken for enrichment and subsequent analysis. In the case of unknown or unpredictable samples, it is advisable to proceed slowly from low sample volumes to higher sample volumes in order to avoid overloading the GC-IMS.

Alternatively, the plant parts of interest may be enclosed with inert material (e. g. frying hose) and the samples are taken from the enclosed volume. This static headspace is particularly beneficial for plant parts that are difficult to reach or for low emitting parts (e. g. foliage), as volatile compounds accumulate in the enclosed gas space. Under laboratory conditions, higher concentration of pVOCs compared to environmental measurements were expected. Therefore, the valve opening times of the PreConcentrator could be reduced with no influence on ion mobility and retention time, but leading to a reduction of signal intensity, and thus avoiding GC-IMS overload. The particular setup with regard to sensitivity was adapted to the specific plants and measurement conditions. Blank measurements and background measurements are conducted prior to sample measurements, to identify pre-concentrator background and background signals from ambient air.

Based on the procedure described above, over 40 samples of typical plants or plant parts, respectively, where already analysed. This included commonly distributed plants such as Elder (Sambucus nigra), beech (Fagus sylvatica), maple (Acer pseudoplatanus), hornbeam (Carpinus betulus), birch (Betula papyrifera), different species of rowan (Sorbus spp.), walnut (Juglans regia), cornel (Cornus mas), different species of cherry (Prunus spp.), apple (Malus sylvestris), Scots pine (Pinus sylvestris), snowdrop (Galanthus spp.), ramps (Allium ursinum), dandelion (Taraxacum spp.), daisy (Bellis perennis), daffodil (Narcissus), crocus (Crocus), lilac (Syringa vulgaris), herb-rober (Geranium robertianum), creeping thistle (Cirsium arvense), clover (Trifolium spp.) and greater stinging nettle (Urticia dioica). Furthermore, differences in floral scents within individual genera were investigated using the example of Narcissus.

IONysos, a custom-made software developed by ION-GAS GmbH, Dortmund, Germany is used for processing, evaluation and visualization. In a first step, drift times are normalized to obtain the relative ion mobility and a gaussian smoothing algorithm is applied to reduce noise. Next, the retention times of each measurement are aligned on the basis of ubiquitous occurring signal with known retention time, allowing comparison between different measurements. This is necessary, as small changes in gas flow can lead to small fluctuations in retention time. After blurring, normalization of drift time and an alignment of retention times, emission patterns are visualized as two-dimensional heatmaps. Substance-specific signal position is determined by the retention time in the GC and the relative ion mobility from the IMS. The peak volume of individual signals permits semi-quantitative statements of the abundance for individual compounds (Figure 5). Individual signals can now be selected manually.

Figure 5.

Exemplary visualization of the raw data as a 2-dimensional heat map (left) and in a 3-dimensional plot (right) using the example of the emission pattern of green daffodil (Narcissus viridiflorus). Three main components could be identified based on the retention time and the relative drift time as eucalyptol (1), benzyl acetate (2) and phenethyl acetate (3).

Substances are identified by a comparison of the retention times and relative ion mobilities with those of reference substances in our database. Unidentified substances are characterised by retention time and relative ion mobility and the peak data is stored for future identification. Peak volumes of the signals detected can be calculated across all measurements for semi-quantitative evaluation. Furthermore, telemetry data (e. g. GC temperature, IMS temperature, gas flow, etc.) is collected in addition to the actual measurements.. This permits (remote-) monitoring of the performance of the instrument.

The collected data is stored in a database based on MySQL (Oracle, Texas), a relational database management system (RDBMS). The database is composed of different tables These tables contain a list of species from the Melbgarten (SpeciesList), a list of reference substances (refcomp), information on sampled plants (samplestatus), emission patterns of sampled plants (emissionpattern) and information on the equipment used (device). These parts are linked to each other via so-called keys, allowing information from different tables to be accessed quickly (Figure 6).

Figure 6.

Scheme of the relational database for plant emission patterns. The database contains various parameters that make it possible to identify the presence of certain signals in species that have already been measured individually.

Individual plant names are recorded with an ID in the SpeciesList table, together with taxonomic information. Data on substances that have already been identified as well as unknown signals are stored in the RefComp table. An ID is assigned to each signal or each substance, respectively with the corresponding retention time and reduced ion mobility. For identified substances, there is additional information on substance name, CAS number, retention index, mass weight and signal type (monomers or dimers).

Device-specific information is stored in the device table. Since only two devices are currently in use, there are only these two entries here.

The last two tables refer to the sample status, which contains information of reference measurements performed (e. g. plant part, plant status, polarity of the measurement, number of identified signals, origin of the sample) and the actual emission patterns recorded. The table emissionpattern contains information about the relative abundance of individual signals in reference measurements, but also information about the nominal signal strength.

Information can thus be extracted from the measurements employing different queries, while the keys guarantee database integrity. The aim is to make this database publicly available. The basic framework and functionality have already been implemented, but the expansion with measurements is still an ongoing work process.

Meteorological data were provided initially by the Meteorological Institute of Bonn University, whose weather station is located in Endenich, approx. 2.5 km from the sampling site. Later on, they were obtained directly from the weather station in the Melbgarten, which was implemented during the AMMOD project. The data considered here include temperature, air pressure, relative humidity, precipitation and global radiation, as well as wind data (wind direction and wind speed).

The year 2022 was an overall warm year with low precipitation. The summer months of July and August, as well as March and April were particularly dry. Even in October and November there were still high temperatures with rather little rainfall. The rainiest month was September, the driest month was August. The coldest month was March, the warmest August (Table 1).

The conditions in the summer of 2021 differed considerably from those of the following year (Table 2), when temperatures were lower and the amount of precipitation higher. This applies in particular to the month of July, with the heavy rain events that occurred there (more than 100 mL m-² in a single day).

Conventional descriptive parameters (e. g. mean, median, standard deviation) are calculated for the eVOC and weather data. Signal intensities are plotted against time for time-resolved immission patterns. Correlations are identified using Pearson correlation coefficient.

Shapiro-Wilk test and Levene’s test are used to test for normal distribution and the homogeneity of the variance. In case of non-normal distribution data, we use the non-parametric Kruskal-Wallis test as significance test and the Wilcox-test for pairwise testing of significant differences between two or more groups. Statistical tests and data evaluation are conducted with the open source software R (version 4.0.4; R Core Team 2023).

Key points in the recording of long-term data are reliable automation and system stability to generate a high throughput of comparable measurements. We are able to achieve an automated measurement for environmental volatiles using GC-IMS by combining in-line enrichment and subsequent GC-IMS analysis. Minimal maintenance is required. However, sensor failure can still lead to occasional measurement gaps. Therefore, the functionality of the device needs to be monitored at regular intervals. Critical parameters for an overview of performance parameters of the GC-IMS are drift, sample and carrier gas flow, detector temperature, GC temperature, preconcentration temperature (precon-chip), Baseline, RIP position and RIP width at half height (WHM).

The mean sample gas flow during the measurement series was 60.74 ± 2.8 mL min^-1 and the mean carrier gas flow was 21.53 ± 0.72 mL min^-1 across measurements for both polarities (Figure 6). Gas flows were largely constant with only minor fluctuations. IMS temperature and the GC temperature were essentially constant (Figure 7). The mean GC-temperature was 79.91 ± 0.61 °C (target value 80 °C) and the mean IMS-temperature was 69.94 ± 0.06 (target value 70 °C). The minimal variations are a result of small fluctuations around the set point and result from technical constraints of the temperature control.

Figure 7.

Telemetry data of gas flow rates, GC and IM temperatures over a selected time period. Carrier gas flow and temperatures in the GC and IMS are very stable. The sample gas flow is subject to greater fluctuations, which can be attributed to the viscosity of air at different temperatures.

The mean enrichment temperature for the PreCon-Chip was 38.1 ± 3.93 °C. Enrichment temperature was slightly sensitive to ambient temperature, especially at high ambient temperatures. This can cause problems, especially in summer. Conversely, the enrichment temperature was broadly stable in low ambient temperature. The mean reduced inverted ion mobility of the RIP was 0.446 ± 0.002 Vs cm^-2 for negative polarity and 0.487 ± 0.002 with a mean WHM of 0.20 ± 0.01. The baseline was constant at 4.97 V. We thus find that all critical parameters are reasonably constant around the target values. Ion mobilities and retention times can be aligned to known signals and a comparability of measurements over longer time periods is ensured.

We were able to record emission data for different plants and observed differences between the detected pVOC patterns. However, there were also plants where we could not detect volatile compounds under undamaged conditions (e. g. Trifolium pratense or Convolvulus arvensis). Individual substances could be identified on the basis of the retention time and the relative ion mobility (Figure 8).

Figure 8.

Emission patterns of different plant species or plant parts, respectively, showing characteristic emissions of certain compounds.

There are distinct differences in the emission patterns of flowers and vegetative tissues. Flowers in particular stand out due to their diversity and, in some cases, high intensity of individual signals. Benzaldehyde, benzyl alcohol, (E)-β-ocimene, methyl salicylate, linalool, methyl benzoate or lilac aldehydes could be identified as typical components of floral odours. We also found as yet unidentified substances e. g. in different species from the genus Sorbus (Figure 9). Unique combinations of substances were often observed in the flower fragrances, even if individual substances are very common. There can be also large differences in the overall signal intensity. Some flowers have no, or only weak emissions (e. g. Galanthus). Others, such as Sambucus nigra or Ligustrum vulgare, have strong emissions. Emission patterns of flowers can be species-specific, as we have shown for species from the genus Narcissus. However, there is often signal overlap in and sometimes differences are only seen in nuances such as the absence or presence of individual specific substance or a different abundance of the same substances (Losch et al. 2023).

Figure 9.

Comparison of emission pattern. (A) Characteristic patterns of different plants and plant parts, respectively. (B) Comparison of leaf emission patterns of Fagus sylvatica for undamaged and damaged leaves.

For vegetative tissue or green plant parts (e. g. leaves), respectively, little or no emissions where observed under undamaged condition and there is hardly any diversity in the composition of the substances. 6-methyl-5-hepten-2-one (e. g. Fagus sylvatica, Acer pseudoplatanus, Sambucus nigra, Quercus robur etc.) or (Z)-3-hexenyl acetate (Urtica dioica) were detected as characteristic substances of green plant parts. Pinus sylvestris represents a notable exception with several typical monoterpenes such as α-pinene, camphene or β-pinene detected. These common components of essential oils are widespread in, e. g. conifers. Damaged leaves, on the other hand, show significantly stronger emissions of substances such as (Z)-3-hexen-1-ol or (Z)-3-hexenyl acetate. In Sambucus nigra, even aromatic substances such as benzyl alcohol were emitted by damaged leaf tissue. In general, a comparatively small number of common and widespread substances are released from mechanically damaged leaf tissue.

During the first measurement phase in the summer of 2021 (n = 539 for positive ions and n = 541 for negative ions), a total of 64 positive ions and 32 negative ions were detected, amounting to a total of 96 unique signals. The measurement series for 2022 was noticeably longer (n = 2152 for positive ions and n = 1913 for negative ions) and 146 unique signals could be detected. The overall number of ambient VOCs present is likely higher, but some signal overlap compromises resolution. Despite these limitations, considerable sensitivity and selectivity could be demonstrated for both negative and positive ions. For positive ions, we observed some superimpositions for highly volatile compounds (retention time < 50 s), due to similar characteristics in retention time and ion mobility. Some of these signal clusters could not be resolved. Many short-chain carbon bodies (< 6 C-atoms) such as ethanol, ethylene, acetone, methanol and also the ubiquitous isoprene are included in this region. Separation was sufficient to identify individual signals at retention times > 50 s. Negative ionisation provides overall fewer and weaker signals, but selectivity is higher due to the lower degree of superimposition. Only very few substances are retrieved after 300 seconds, with the last peaks observed at ca. 700 s (both polarities). While we were not able to identify substances in the negative range, we could identify 15 substances amongst the detected signals for positive ionisation, including monoterpenes (MTs) such as α-pinene, β-pinene, camphene, DL-limonene, p-cymene, (E)-β-ocimene and Eucalyptol, as well as aromatic compounds (e. g. Benzaldehyde, Anisole, p-cresol) and aliphatic compounds such as 6-methyl-5-hepten-2-one or (Z)-3-hexen-1-ol. Surprisingly, we failed to identify any typical anthropogenic compounds (e. g. xylene or toluene). Floral volatiles were also less strongly represented than expected.

The data obtained in this way contain information on changes over time such as weekly averages for the sum of signal intensities, but also for individual substances (Figure 10). The sum of the signal intensities reveals information about the general course of variations, while individual substances give a more fine-grained insight of seasonal variation. For 2021 and 2022, the temporal course of total signal intensity partly reflects the prevailing weather conditions, with the highest signal intensities in June (warmest month). For monoterpenes such as α-pinene we observed deviating temporal courses with irregular spikes.

Figure 10.

Time resolved data as weekly average for the total signal intensity (Positive & negative ions) and α-pinene for the period of 2022.

In general, the measurement period of 2022, covers a wide range of seasonal variation and thus allows a more detailed view, compared to the rather short series of 2021. June was again the month with the highest average signal intensities. However, patterns across the summer months are less divergent, which may be due to the overall less variable weather patterns in 2022.

The data can also be utilized to look at weekly or diurnal variation, respectively. In principle, intra-week variation could be an indication of the anthropogenic origin of substances, because emission of air pollutants tends to be higher on weekdays (e. g. NO_x and VOCs from traffic) Especially for the first period (March/April) it seems that there are differences between weekends and weekdays (Figure 11). We observed no statistically significant differences in concentration across the weekdays (p > 0.05). This applies to the sum of all signals, but also to individual substances such as α-pinene, indicating an essentially non-anthropogenic, i. e., biogenic origin of the substances detected.

Figure 11.

Weekly variation of total signal intensity and α-pinene during three seasonal intervals for 2022.

The diurnal changes can provide information about the emission behaviour of plants. Fine-grained analysis revealed that signal intensities across both polarities display significant diurnal patterns, which is particularly pronounced during the main period of activity from May to August 2022 (Figure 12). Overall signal intensity reaches a maximum in the afternoon and a minimum in the early morning hours (χ² = 65.615, p < 0.001). Conversely, for some compounds, such as α-pinene, we observed a significant diurnal variation (χ² = 114.38, p < 0.001) with a maximum in the early morning hours and a minimum towards the afternoon.

Figure 12.

Diurnal variation of total signal intensity and α-pinene within the three seasonal intervals for 2022.

Correlations with abiotic factors can also be investigated, in addition to time patterns. A comparison of pVOC data to meteorological data is relevant since meteorological factors are expected to affect pVOC-emissions.

Over the entire measurement period of 2022 (n = 2152), we obtained two more or less separate clusters with regard to the correlation to temperature, which reflect the abovementioned seasonality quite well. Almost all substances show a positive correlation to temperature across the year, which is reflected by total signal intensity. We find a strong positive correlation (Pearson correlation) to temperature, with R = 0.57 (p < 0.001) for the total signal intensity for positive ions and with R = 0.57 (p < 0.001) for negative ions (Figure 13). However, the positive correlation between temperature and concentrations is less pronounced for monoterpenes such as α-pinene (R = 0.35, p < 0.001). We only have good data for June, July and August (n = 539) from the year 2021. Here, significant correlations could also be observed with R = 0.67 (p < 0.001) for the total signal intensity. Again, α-pinene showed a considerably weaker correlation with temperature (R = 0.27, p < 0.001).

Figure 13.

Correlation of total signal intensity and signal intensity of α-pinene with temperature for the measurement period of 2021 (A) and 2022 (B).

Overall, we achieved good results in terms of the robustness of the measurements. The measurements were automated and maintenance was minimal, in particular as remote quality control was available. Ion mobility and the retention time from gas chromatographic pre-separation are essential for the identification or comparability of substances across a series of measurements. These parameters are heavily influenced by the temperature of the GC pre-separation, detector temperature and carrier gas flow. It was therefore important that these parameters were successfully kept sufficiently constant over the course of the measurement series. The automated normalization of ion mobility to the RIP (or RIN for negative polarity) also worked excellently and no deviations were observed.

Fluctuations occurred mainly for the enrichment temperature of the PreCon-chip. For the adsorption of VOCs, the chip is cooled to approx. 40 °C using air cooling. Especially at very high and very low ambient temperatures, a deviation from the target temperature can occur, as the temperature gradient is no longer sufficient for cooling. This is reflected in an enrichment temperature of ~ 50 °C during the hottest days in June (ambient temperature 39 °C). In theory, a higher enrichment temperature is associated with lower performance for adsorption. However, we observed no drops in signal intensity due to high temperatures, indicating that sufficient adsorption of volatiles also takes place at higher temperatures and/or that the effect of increased emissions of volatiles at higher temperature overcompensates the effect of reduced adsorption efficiency. In order to further improve reproducibility of the measurements, a cooling system could be implemented in the station in future.

Slight fluctuations were also observed in the sample gas flow. This is related to the viscosity of gases at different temperatures and air pressures. As the sample volume is fixed to 1000 mL, this factor only affects the duration of sample collection and should not affect signal intensity. In summary, the continuous stationary field measurements can be considered a complete success in technical terms.

We have here demonstrated for the first time a novel method for automatic long-term monitoring of volatile organic compounds in environmental air using a ppq-tec-GC-IMS as alternative to existing monitoring techniques such as PTR-MS, zNose® or relaxed eddy accumulation (Tholl et al. 2006; Sarkar et al. 2020b; Sarkar et al. 2020a). We were able to fully automate data collection with minimal maintenance and without additional consumables (e. g. adsorbent). The pre-concentrator chip permits in-line sampling and analysis at a high level of automation and with good time-resolution (one measurement per hour). The high sensitivity obtained by the in-line enrichment was sufficient to detect a total of 146 signals (109 in the positive ion mode and 37 in the negative mode).

The actual number may well be higher, but a clear assignment of additional individual signals is difficult, especially for highly volatile substances, due to superimpositions. The technique, as currently implemented, thus reaches its technical limits especially for substances with < 6 C atoms (including isoprene). The time resolution of the measurement permits the identification of seasonal changes and a correlation to abiotic factors (weather) and in the long-term even interannual trends. Furthermore, it was possible to record time-resolved changes of pVOC-imission and observing even more fine-grained diurnal variation of certain signals.

Based on the parameters (retention time and ion mobility), it was possible to identify certain signals that can be directly assigned to a biogenic or plant origin, respectively. These substances comprise typical plant monoterpenes such as α-pinene, β-pinene, camphene, DL-limonene and eucalyptol. In addition, aromatic compounds such as benzaldehyde or p-cresol, but also aliphatic compounds such as 6-methyl-5-hepten-2-ones were identified. The location of the monitoring station and the fact that we found concentrations to be essentially uniform across the week, suggest that the vast majority of volatiles detected can be considered as biogenic. However, the substances that could be identified represent more or less ubiquitous plant volatiles and cannot at present be assigned to any particular plant species or plant group. Eucalyptol represents a notable exception and can be assigned to the emissions of Juglans regia at our specific sampling site with reasonable certainty.

Furthermore, anthropogenic air pollution, in particular from traffic, would usually be expected to have a significantly lower impact at weekends (Blanchard et al. 2008). The same applies to seasonality, where anthropogenic emissions from heating and transport would be expected to increase during the colder months. It remains unclear whether this would lead to lower intensities of certain VOCs due to less input into the environment, or to an increase in all signal intensities due to lower NO_x input from e. g. traffic. However, neither case could be observed.

We have first promising results regarding the seasonal and diurnal variation of volatile organic compounds in ambient air, permitting the identification of trends. Many signals have not so far been identified as to chemical identity - the identification of additional signals and their inclusion in the reference database are therefore certainly major issues that need to be addressed in future. Due to the nature of GC-IMS, the physical signal parameters also permit a future identification of currently unidentified substances. An expansion of the sampling of plants from the target area to identify characteristic emitters will hopefully reveal characteristic source species for particular signals, permitting a clearer biological interpretation of the patterns observed.

However, one of the most important points should be the evaluation of the raw data. At present, the spectra are visually revised and the signals segregated and identified. With more than 4000 measurements per year from a single sampling locality, the volume of data generated is enormous. If this is extrapolated over several years, and possibly with an increasing number of stations, a visual approach for comparing individual emission patterns does not appear to be expedient. We are therefore planning to automate pattern recognition in cooperation with AI specialists.

The statistical analysis of the selected data only scratches the surface of what is possible. In fact, the interplay of abiotic and biotic factors can involve a complex network of interactions (Figure 14). Anthropogenic substances, e. g. from traffic or agriculture, also influence the composition of volatiles. Of course, above all this are the prevailing weather conditions, which influence both biogenic and anthropogenic inputs, but also the distribution of these (through wind). Therefore, more sophisticated statistical analyzes and models should be considered in the future in order to obtain more information from the data.

Figure 14.

Various influencing factors that can affect measured VOC concentrations.

GLV Green leaf volatiles

VOC Volatile organic compound

pVOV plant volatile organic compound

eVOC environmental volatile organic compound

IMS Ion mobility spectrometry

GC Gas chromatography

MEMS Microelectromechanical system

MS Mass spectrometry

ppb parts per billion

ppt parts per trillion

Collection sites for the airborne traps should be in an open habitat, as bushes and trees, as well as boulders, buildings, streets and railroads in the near vicinity can lead to air turbulation and will affect the pollen collection. The trap should be equipped with a built-in spirit level and height-adjustable feet to level the device even on uneven ground and make it easier to turn depending on the wind direction. When a trap is not able to rotate in the direction of the wind, it results in non-optimal catch conditions, which influence the comparability of the results. The trap is designed to operate in all weather conditions in the field and can be powered by a solar panel and battery. However, the currently available solar modules do not yet guarantee year-round continuous operation and therefore require connection of the trap to a power grid. The control module and the battery are located in a lockable compartment of the trap, where a GPS tracker could also be attached. To reduce rodent infestation, all openings were reduced in size compared to the original prototype.

We made changes to the design of the Hirst spore trap (Hirst 1952), which has been used to monitor airborne pollen across Germany by the Stiftung Deutscher Polleninformationsdienst (PID) since 1983. The newly designed pollen trap, called the ‘A1 Volumetric Air Sampler’ (Khan et al. 2022; Figure 1), can work independently in the field for an extended period as it collects pollen in 24 microcentrifuge tubes with adjustable time intervals. The 0.2 ml microcentrifuge tubes are clicked into a forward and backward rotating platform that can switch from one tube to another by electronic timing (Figure 2). This allows interval flexibility, and sampling can be carried out over hourly, daily or weekly intervals. In addition, the time at which the fan actively sucks pollen into the trap can be time-controlled.

Figure 1.

The A1 Volumetric Air Sampler.

Figure 2.

Interior of the A1 Volumetric Air Sampler, with the cover removed, revealing the carousel for 24 sample tubes (Photograph: Gulzar Khan, test site Britz).

The sampling intervals for the wind-dispersed pollen was coordinated with the sampling of the Malaise (insect) trap to compare airborne pollen with the pollen transported by insects (fan suction time: 45 min/h; fan cycle: 75 %; fan duration: 1350; day mode only; sampling period: 7 days per tube).

DNA extraction and PCR were performed in a laboratory designed to minimize external contamination. Best practice recommendations for highest quality assurance in the laboratory are the following:

• Laboratory protocols should be conducted in clean rooms, which are entered via an air lock. The clean rooms are equipped with an air purification system (H13 particle filter) and the air is exchanged every 7 minutes. UV light should be used regularly, preferably every night, and between each DNA extraction and PCR run for at least 30 minutes to prevent potential DNA contamination in the laboratory environment (Figure 3).
• Due to the frequent use of UV light in the laboratory, machines and devices should be covered with a UV-resistant film to protect the plastic parts of the instruments. DNA extraction and PCR should not take place in the same room, and separate equipment should be provided for both laboratories without transporting any of these between the two areas. In addition, all possible steps of extraction and PCR should be conducted under a laminar flow cabinet (Figure 3).
• Sample processing should be conducted with DNA-free equipment. All surfaces should be regularly cleaned with a decontamination solution (e. g. DNA AWAY, Carl Roth GmbH + Co. KG, Karlsruhe, Germany) after each use. Certain steps of the DNA extraction, and PCR should be performed using filter tip pipettes. We recommend to take multiple field, extraction, and PCR blanks for quality control and establishment of false positive thresholds.
• Personal protective equipment: We recommend the use of disposable ME DI-INN overalls (Carl Roth GmbH + Co. KG, Karlsruhe, Germany), shoe covers (VWR, Radnor, USA) and disposable gloves (Th. Geyer GmbH & Co. KG, Renningen, Deutschland; Figure 3). The disposable gloves are changed several times while working in the laboratory, and the disposable overalls and shoe covers are replaced after each day.

Following removal of insects, the preservative ethanol from the Malaise trap samples were vacuum filtered using a cellulose nitrate (CN) membrane (GVS Filter Technology, Sanford, USA; diameter 47 mm and 0.22 µl pore size, Figures 4, 5). After filtration, the CN filter was cut in half and each part was placed in a separate 2 ml SafeSeal micro tube (Sarstedt AG & Co. KG, Nümbrecht, Germany). One part of the filter paper was used for DNA isolation, the other part can be used as voucher or for an additional DNA-isolation. Short term storage is recommended at a temperature of −20 °C until further use. We recommend −80 °C or cryo-storage for long-term storage of vouchers or DNA.

Figure 4.

Vacuum filtration system for the preservative ethanol from the Malaise trap insect samples. (Compare with Figure 5).

Figure 5.

The cellulose nitrate membrane following vacuum filtration (see Figure 4).

DNA isolation was performed with the NucleoMag Plant kit (Macherey-Nagel, Düren, Germany) for all samples and two DNA extraction blanks. Prior to DNA isolation three sterile tungsten carbide beads (Qiagen, Hilden, Germany; diameter 3 mm) and 600 µl of lysis buffer MC1 were added to the filter paper in the microcentrifuge tube. These samples were disrupted in a Retsch MM400 bead mill (Retsch GmbH, Haan, Deutschland) for 2.5 minutes at a frequency of 30 Hz. 10 µl of Proteinase K (20 mg/ml) and 5 µl of RNase A were added to the ground material and the samples were incubated at 60 °C for 45 minutes with constant shaking in a ThermoMixer C (Eppendorf, Hamburg, Germany; 400 rpm). The incubation step was followed by centrifugation of the samples (12,000 × g; 10 min). 400 µl of the cleared lysate were transferred to a clean 2 ml microcentrifuge tube and 400 µl of binding buffer MC2 and 10 µl of magnetic beads were added. In the further steps of the manufacturer’s protocol, only 25 % of the specified amounts of MC3, MC4, 80 % ethanol and MC5 were used. The washing step with ethanol was performed twice. 35 µl of MC6 elution buffer (55 °C) was added to the samples and incubated at room temperature for 5 minutes. Up to 30 µl was removed after application to magnets for a final elution volume.

For DNA isolation of pollen from air, we recommend the use of the DNeasy PowerSoil Pro Kit (Qiagen, Hilden, Germany), which seems to overcome the humic acid components as well as other PCR inhibitors contained in these sample types. Some changes to the standard protocol are recommended. The Power Beads provided in the kit were transferred from the Power Beads Pro Tubes to the 2 ml micro tube containing the airborne pollen. This step is necessary because it is not possible to transfer the collected pollen directly. After adding 800 µl of the lysis buffer CD1, the tubes were placed in a Retsch MM400 bead mill for 2.5 minutes at a frequency of 30 Hz. A subsequent centrifugation step was carried out at 15,000 × g for 5 minutes. All other steps were performed as described in the manufacturer’s protocol, except for the elution step; here only 50 µl of the C6 solution was used.

Several publications have addressed the quality of plant specific barcoding regions (CBOL Plant Working Group 2009; Hollingsworth et al. 2011; Li et al. 2014; Kolter and Gemeinholzer 2020). Plastid markers like matK, rbcL, trnH-psbA and others are mostly uniparental inherited in the plant kingdom and have comparatively slow mutation rates (Birky 1988) and therefore primarily provide higher taxonomic identifications (Vijayan and Tsou 2010). The rbcL-gene currently has the best taxonomic coverage in reference databases (Kolter and Gemeinholzer 2020); however, many sequences are without a voucher and limited metadata information, which is not the gold standard for metabarcoding purposes. The non-coding plastid DNA regions often vary in fragment length throughout the plant kingdom (Palmer et al. 1988; Clegg et al. 1991) and comprehensive reference databases have not yet been established.

Here, we used the nuclear internal transcribed spacer region (ITS) for plant identification which has strongly conserved primer binding sites in adjacent mRNA genes. Kolter and Gemeinholzer (2021) and others showed that this region between the 18S, 5.8S and 28S mRNA genes provides a high probability of correct metabarcoding identification at the genus or species level throughout the plant kingdom. However, because it is a bi-parentally inherited multicopy region of the nuclear genome that is usually subject to concerted evolution, identification could be ambiguous in hybrids, polyploids, and apomictic species complexes, as well as in recently strongly radiating groups. Recommendations are to use both the ITS1 as well as the ITS2 region with the relatively uninformative 5.8 mRNA gene in between for plant identification; however, Illumina MiSeq sequencing only covers regions up to 350 bp (e. g. China Plant BOL Group et al. 2011; Kolter and Gemeinholzer 2021). Therefore, we recommend using either the ITS1 or the ITS2 region. The ITS1 region is very informative but varies in length, especially for gymnosperms, and therefore cannot be used for airborne pollen. For this reason, we used the ITS2 region for airborne pollen as well as Malaise trap plant trace identification. The ITS2 reference database has good species coverage, but with a large proportion of single individuals with no geographical coverage of genetic diversity. Kolter and Gemeinholzer (2021) tested various proposed primer combinations on mixed mock communities and evaluated the identification ability of ITS2. Following the recommendations of these authors, we used the following primer combinations: ITS-3p62plF1 (ACBTRGTGTGAATTGCAGRATC) and ITS-4unR1 (TCCTCCGCTTATTKATATGC).

PCR allows for the amplification of specific barcoding regions as well as tagging specific samples with unique labels, as metabarcoding is achieved via parallel sequencing of multiple samples in one run (Lundberg et al. 2013; Schnell et al. 2015). The primers carry a long nucleotide tail, which consists of a flow cell binding adapter, a binding site for sequencing primers and a label for sample identification. These fusion PCR primers can be applied to the samples using one-step or two-step PCR (Lundberg et al. 2013). The 2-step PCR is the most cost-effective multiplexing method and has higher consistency and taxa detection efficiency than the 1-step approach, although PCR always introduces bias in the laboratory step, making quantity estimation unlikely. Nevertheless, the 2-step PCR has an excellent sensitivity to PCR-inhibiting substances (e. g. humic acids and tannins), since performing the PCR twice results in a dilution of these inhibiting substances. The advantages of 1-step PCR are less time and less susceptibility to cross-contamination and a reduced PCR bias. Therefore, this approach is suitable for metabarcoding of clean samples where the maximum number of species is not relevant. For more detailed and larger biodiversity assessments, 2-step PCR is a better choice (Zizka et al. 2019), which is why we recommend using it.

PCR was performed with three replicates per sample. Two DNA extraction blanks and two PCR blanks were added to every 96 well PCR plate. The Canadian Centre for Barcoding PlatinumTaq Protocol (Ivanova and Grainger 2007) was used, with the addition of 0.3 µl of BSA (10 ng/µl) and 1.25 µl of 50 % DMSO in a total volume of 12.5 µl per reaction. Initial denaturation was performed at 95 °C for 3 minutes, followed by 30 cycles of denaturation at 95 °C for 30 seconds, annealing at 50 °C for 30 seconds, and extension at 72 °C for 45 seconds. The final extension was performed at 72 °C for 8 minutes.

After PCR, 5 µl of each of the three PCR replicates of a sample were combined to a total volume of 15 µl and purified with 1.5 µl Exonuclease 1 (Thermo Fisher Scientific, Waltham, USA) with 37 °C incubation for 30 minutes followed by 80 °C incubation for 15 minutes. Illumina sequencing was performed at LCG Genomics GmbH (Berlin) on a MiSeq (2 x 300 bp) platform with 12 additional cycles. For the additional PCR cycling MyTaqTM Red Mix polymerase (Meridian Bioscience, Cincinnati, USA) was used and it consisted of three cycles with a low annealing temperature (15 sec 96 °C, 30 sec 50 °C, 90 sec 70 °C), followed by nine cycles with increased annealing temperature (15 sec 96 °C, 30 sec 58 °C, 90 sec 70 °C).

We recommend double-stranded sequencing on a MiSeq or NextSeq platform (2 × 300 bp, Illumina, San Diego, USA), depending on the amount of parallel sequencing and sequence read depth. This platform has currently the highest sequence accuracy and quality score. Long reads can also be performed with a MinION-tool (Oxford Nanopore Technologies, Oxford, United Kingdom) which has the benefit of being small and transportable, but due to the high error rates, species level identification via metabarcoding is currently complicated. Another long-read sequencing platform is the Revio or Sequel platform (Pacific Biosciences, Menlo Park, USA). This PacBio sequencing platform has a shorter sequencing time than the MinION, but the sequencing equipment is very expensive, and the sequences have high error rates. In addition, this sequencing platform is significantly larger than the MinION and can therefore not be taken into the field (Hu et al. 2021). Innovations in sequencing technology could lead to further developments in this area.

The sequencing data were demultiplexed by index sequences into three different files, and primers and the index sequences were trimmed with Cutadapt (Martin 2011). Then, the sequences were quality-filtered with FastQC (Andrews 2012). After quality filtering, the R package DADA2 (Callahan et al. 2016) was used to estimate the error rate, followed by denoising with the error profile using a pseudo-pooling function. Forward and reverse reads were merged by using DADA2 (Callahan et al. 2016). Chimeras from denoised and merged read pairs were removed using the removeBimeraDenovo function of the DADA2 package (Callahan et al. 2016). The resulting amplicon sequence variants (ASV) table was used for ASV identification by comparison with the PLANiTS2 database (Banchi et al. 2020), performed with the DADA2 assignTaxonomy function (Callahan et al. 2016). Currently, the PLANiTS database is the most comprehensive for curated, reliable and updated ITS reference sequences. However, this may change in the future as initiatives (e. g. such as NORBol and ABol) aim to create national reference databases. Fungal contamination was confirmed and removed by performing a local BLASTn search (Camacho et al. 2009) using the plant sequences and a custom fungal ITS-BLAST database. The final species list was thoroughly checked, to remove potential laboratory contaminations or wrongly assigned sequences. ASVs with ambiguous species identifiers received only genus-level identifiers. ASVs from the DNA extraction and PCR blanks were used to calculate a relative abundance, ASVs below this threshold were removed.

In times of climate change and biodiversity declines the protection of our ecosystems is of major importance. Insects, one of the most diverse groups on earth, are indispensable for the function of ecosystems as they provide fundamental services such as pollination (Badenes-Pérez 2022), decomposition of biomass (Marschalek and Deutschman 2022) and pest control (Campbell et al. 2017). Therefore, data on composition, shifts in the structure or decline of local insect communities allow us to draw conclusions about the health of the associated habitats (Brown 1997). Permanent, long-term biodiversity monitoring sites could provide those long time-series and serve as early-warning systems for a decrease in ecosystem health. However, traditional biodiversity surveys are time consuming, costly and highly human resource intensive. In Germany, a dramatic loss of insect diversity advanced unnoticed for many years due to the lack of a thorough insect monitoring (Hallmann et al. 2017).

Metabarcoding, a genetic approach developed in the last 10 years with the emergence of next-generation sequencing, enables the rapid identification of several thousand of specimens in parallel, partly circumventing the increasing shortage of taxonomic experts. Despite these technological advances in the lab, permanent biodiversity surveys still continue to require human intervention to capture specimens for monitoring on a regular basis. Many of the traditionally used insect traps like Malaise traps, vane traps and pitfall traps are usually emptied in a bi-weekly cycle, depending on weather conditions, season and chosen fixative, to ensure specimen quality. Especially in rural areas or in extreme environments, which are often only poorly accessible, permanent insect surveys are often excessively difficult or virtually impossible, unless they rely on a team of local volunteers to empty the traps. Here, we present a prototype for an automated insect trap which can sample up to 12 bulk samples without human intervention.

As a passive flight interception trap, the Malaise trap has a broad target spectrum, including all insects actively flying through the habitat but also some ground dwelling taxa (Geiger et al. 2016). Besides others, the most commonly found taxa in Malaise trap catches from Germany are the Diptera, Hymenoptera, Lepidoptera, Hemiptera and Coleoptera (Geiger et al. 2016; Kirse et al. 2021). The first Malaise trap was designed by the Swedish Hymenopterist René Edmund Malaise in 1934, based at the Riksmuseum in Stockholm (Malaise 1937). Over the years he modified the structure and design of the trap several times to allow for new arising purposes (Uhler et al. 2022). Today, several types of Malaise traps are available, which differ in colour and mesh size (Uhler et al. 2022). One of the most commonly used Malaise trap types is the bi-colored Townes trap (Townes 1962; Matthews and Matthews 1983). The set-up resembles a white mesh tent which is separated into two halves along its central axis by a black mesh panel. The black mesh functions as a barrier for flying insects, which try to escape by flying upwards towards the brighter white mesh roof until they reach the apex of the trap. The apex consists of a reversed bottle (from here on referred to as trapping bottle) which the insects enter through a 10 cm² side opening. From here, insects are directed towards a collection bottle, which is connected to the lower outlet of the trapping bottle. The collection bottle is filled with a preservative fluid (e. g. ethanol or propandiol) to ensure the best possible preservation of the morphology and DNA of specimens. Depending on habitat and insect activity the collection bottle should be exchanged every one to two weeks.

The automated AMMOD multisampler replaces the traditionally used collection bottle. The AMMOD multisampler has a housing measuring 66 cm × 66 cm × 35.5 cm which is mounted onto a tripod stand (Nedo, heavy-duty Aluminium Tripod, Ref.-No. 200204). The height of the tripod stand must be manually adapted to the height of the apex of the trap, which should be around 180–190 cm above ground. Here, we used a Malaise trap which was designed by the Entomological Society Krefeld (Hallmann et al. 2017), and is very similar to the bi-coloured Townes model (Townes 1962) but differs in two important features: First, the black mesh is slightly extended so that the Malaise trap can be tightly anchored to the ground, preventing insects from escaping between ground and mesh (Hallmann et al. 2017). Secondly, the trapping bottle is at a slightly angled position to lower the risk of water running into the collecting bottle on rainy days. To connect the AMMOD multisampler with the Krefeld Malaise trap, a 3D printed plug-in connection adapter sits on top of the AMMOD multisampler box (Figure 1). The counterpart adapter can be screwed onto the trapping bottle (S65 thread) before being plugged onto the connection adapter.

The AMMOD multisampler consists of six subsystems: (1) the rotary mechanism, (2) the user interface, (3) the external sensors and modules, (4) the stopping mechanism, (5) the energy supply and (6) the microcontroller (Figure 2).

Figure 1.

Setup of a Townes Malaise trap (Krefeld model) equipped with an automated AMMOD multisampler.

Figure 2.

The electronic set-up of the AMMOD-multisampler consists of six subsystems: (1) rotary mechanism, (2) stopping mechanism, (3) external sensors and modules, (4) user interface (5) energy supply and the microcontroller which connects all subsystems.

The code of the controller is separated into 4 main sections (Figure 3): (1) Core functions, (2) Data log, (3) Sampling programs and (4) human machine interface (HMI).

Figure 3.

Flowchart visualizing main code functionalities.

• The core function consists of the global functions, declarations, and libraries which are used throughout the code and determine the most basic functionalities of the controller such as display refresh rate, turning on/off the display when there is no human interaction etc.
• Data logging includes the functionalities of the sensors, the communication module, data logging and error checking. This part is responsible for monitoring the system health, keeping a record of all events and notifying the user in case of unwanted events.
• Sampling Programs consists of the Manual and Automated Sampling programs, the controlling of the rotary mechanism including start/stop commands for the motor driver, determining the current position and determining if it is the correct sampling unit.
• Human Machine Interface consists of the input and output components. The input components are keypad and Serial communication. The controller constantly checks if there are user inputs. The output component is the display. In contrast to the input components, it does not constantly update the information rather it waits for the user input.

Additionally to the controller code, there are two more codes which are used to assist the development.

• The first is an I2C address finder for the components that are used. Based on the manufacturer, addresses of the same components might be different. For this reason, it is good practice to check the component address and respectively change it in the controller code.
• The second is the Modem and SIM card activator. This code enables the modem and binds it to the SIM card. Additionally, it provides information regarding the SIM card provider which has to be embedded into the controller code.

To allow for a wider variety of research questions, but also to monitor the system itself, the AMMOD multisampler is equipped with several sensors. The sensors can be divided into two categories: environmental sensors and system sensors.

Next to a real-time clock (RTC), which provides the actual time and date, the system comes with an SD card and a communication module. The SD card module equipped with an 8 GB SD card is used as a hard disk drive to log information about trap performance in a simple text file (log file). The log file is an important tool as it allows to keep track of sensor data, but also system failures. As soon as the system is turned on, booting information is collected including local time, machine ID, UTC time and geographical parameters (longitude, latitude). With the start of a new sampling cycle (program start) every 5 minutes the measured sensor values but also the in-use collection bottle is logged. Additionally, the log file keeps track of system failures, which will be indicated with an “E” next to any questionable sensor value sensor value.

The implemented communication module functions on European frequencies and consists of a Plug and Play Arduino shield. The shield comes with a GPRS/LTE and GNSS antenna. Like all external modules is the communication module placed inside the metallic AMMOD multisampler box which can reduce signal quality. To amplify the signal, an external antenna is installed outside the AMMOD multisampler and connected via an antenna cable using an adapter connector to connect to the SMA cable extender. The GNSS system can use navigational satellites from other networks beyond the GPS. Here the GNSS module is used to record the geo location but also the exact UTC-time. The GPRS is used to send SMS notifications to the users e. g. in case of an error. After inserting a new SIM card into the shield, the user has to activate the modem and register the network via AT-command prompt.

The main Menu provides four different submenus: ‘Manual Program’, ‘Auto Program’, ‘Config Data’ and ‘Info’. By using the built-in 1×5 keypad the user can choose between menus. The initial screen shows the actual date and time that future programs will be based on. If the provided information is incorrect the user can change settings in the ‘Config Data’ menu. Additionally, the user can retrieve additional information from the system by selecting the ‘Info’ menu. This menu contains information about the AMMOD multisampler-ID and readings of implemented sensors: air temperature, air humidity, soil temperature, soil moisture, light intensity.

The ‘Auto Program’ menu provides two built-in standard programs which the user can choose from: (1) ambient illuminance program and (2) soil temperature program. Collection bottles will be turned under the trapping bottle depending on either measured illuminance or soil temperature. Thus, the trap can be used to monitor insect activity patterns depending on abiotic factors (Table 1). The standard settings can easily be changed in the program code and adapted to the required purposes. After selecting the desired built-in program, the user will be asked to provide the planned sampling starting date and time.

Next to built-in programs users have the possibility to design their own time-controlled program in the ‘Manual Program’ menu. This way, the sampling interval can be easily adapted to the respective research question.

First the user must enter the planned starting time and date. In the next menu the user is prompted to enter the total number of intervals (number of bottle changes) including pauses (position 13). In the last step each interval must be exactly defined in terms of duration lasting from 15 minutes to up to 99 days. After entering data for the last interval, the system will turn position 13 under the trapping bottle before starting with the program at the defined starting time.

Depending on the number of collection intervals, defining the sampling program manually can be very time consuming. With the Graphical User Interface (GUI) ArduinoGUI.exe a desired sampling program can be defined in advance. The GUI is only available for windows machines on which the Arduino IDE is installed. To use all functions of the serial communication, the microcontroller must be connected to the windows machine using the USB ports of the two systems. Like the ‘Info’ and ‘Config Data’ menu of the in-built manual control system, the user can now read information but also change data via serial communication by clicking on ‘Info’ and ‘Config Data’ respectively (Figure 3). Additionally, the ‘Create Program’ menu of the GUI interface can be used to generate a program file (.txt format) in advance. The text file contains a data string consisting of the defined starting date and time, number of intervals and detailed information about duration of each sampling interval. The text file can be uploaded to the ArduinoGUI in the ‘Start Program’ menu choosing the option ‘Manual Program’ and transferred to the microcontroller by clicking ‘Transfer Data’ as soon as a connection between windows machine and microcontroller is established. Next to the ‘Manual Program’ option the user can choose the option ‘Automated Program’. In comparison to the built-in manual control system, the user can choose among four different predefined programs, each controlled by a different sensor. In detail the following programs are available: (1) Illuminance Program, (2) Soil Temperature Program, (3) Soil Moisture Program, (4) Air Humidity Program. The automated multisampler will choose the collection bottle according to the measured light intensity (lux), soil temperature (°C), soil moisture (%) and air humidity (%) respectively. Again, the user only has to define the sampling starting date and time, choose the program and click on transfer data. Thus, insects flying at a defined time interval, daylight intensity, ambient temperature or humidity can be collected automatically over longer collection periods.

Figure 1.

Four-channel audio sensor BRITZ02. The ultrasound sensor of BRITZ03 was placed at the tower in the background.

With either program (manual and automated) the system will turn position 13 under the trapping bottle and will be in stand by until the defined starting time.

The sensor components have been selected with the following goals in mind:

• high spatial resolution
• long range
• enough computing power for on-site analysis
• low power consumption

Since the area to be monitored grows quadratically with the range, we decided to use a high-quality sensor array, even though this solution is more cost-intensive than standard audio recording devices. The main advantage of using a single long-range sensor over a larger number of close-range sensors is that significantly less data needs to be stored for virtually the same information. It also facilitates maintenance and data collection in the field.

To obtain a good spatial resolution, we chose a setup with four cardioid microphones arranged in a plane and each microphone oriented in one of the four cardinal directions. The plate above the microphones protects them from rain (Figure 1). Low-noise microphones and audio interfaces enabled recording of audio signals at the greatest possible distance (Table 1). In this study, we operated three four-channel audio sensors, two at Britz (BRITZ) near Eberswalde, Brandenburg, and one at Melbgarten (MBG), Bonn, North-Rhine Westphalia. At Britz we also tested two types of ultrasonic microphones. In addition to the AMMOD sensors, we continuously operated one commercial SM4 (Wildlife Acoustics) stereo recorder each at Britz (BRITZ01) and Melbgarten (MGB01).

Figure 1.

Four-channel audio sensor BRITZ02. The ultrasound sensor of BRITZ03 was placed at the tower in the background.

As a computing plat form we chose the widely used Raspberry Pi (Figure 2). It offers a large community with good documentation and support. The option to run Python supports easy implementation of new features. It also has enough computing power to perform preliminary on-site data analysis. However, a Raspberry Pi 4 with at least 4 GB RAM is required for this purpose. We also recommend using a StromPI 3 with Battery Shield. The expansion shield allows setting date and time after rebooting even without an internet connection. This is mandatory for stable and correct file naming. It also allows bridging short power failures and proper shutdown of the system in the event of a longer power outage. In addition, we added an external USB WiFi adapter (TPLINK TL-WN722N WLAN-Adapter) to connect an external directional antenna. This allows the sensor to be placed at a greater distance from the base station. A robust low-power external 1TB SSD hard disk (SanDisk) was used for storage.

Figure 2.

Audio interface connected to Raspberry Pi.

On site, audio signals are recorded and optionally analysed. The raw data, sensor information, and preliminary results are sent to the base station. Any updates can be made through a remote maintenance connection to the station. A backup of the raw data is also saved on local storage. Depending on the size of the storage and recording schedule, the data is periodically collected and taken to a data centre for further analysis. The in-depth analysis can lead to new insights and improvements, by fine-tuning species-specific detection thresholds or by using larger versions of the classifier models. The results and metadata are uploaded to the AMMOD Cloud (Figure 4).

Figure 4.

Workflow for audio data acquisition, processing, and storage.

6.2.4.1 Software components

Passive acoustic monitoring projects often result in big collections of audio data that are difficult to handle. To address this challenge, a service with two components was developed to make the process more efficient (Figure 5):

Figure 5.

The two components of the Web Service share the database and file system.

• The import script analyses audio recordings using classifiers and enters the results into a database.
• The web service enables the analysis of the imported data via a web interface.

The software components of the service can be found in the Git repository (https://code.naturkundemuseum.berlin/tsa/monitoring-data-analyze-service).

6.2.4.2 Web service

The web service consists of two parts. The front end is accessible via a web browser. It displays all background tasks on the landing page (Figure 6). The left navigation menu lists all imported collections ordered by recording station, year of data acquisition, and classification model used for bird identification. The user can filter them to find the desired collection. All jobs are displayed on the Dashboard view, so the user can see whether the background job is already done or still running.

Figure 6.

Landing page of the web service.

Clicking on one of the entries in the navigation menu switches to the view of a collection (Figure 7). Basic statistics (Stats) as well as eligible query parameters are shown for the selected collection.

Figure 7.

Collection view, showing statistics and eligible query parameters for the selected dataset “BRITZ01: 2019”.

This view comprises five components.

• DB-Index List:

• Selected species can be provided with a database index to speed up queries for these species (Figure 8).

• Stats: This section displays general information about the collection (Figure 9):

• Recording period
• Number of recordings
• Sum of the recording duration
• Number of prediction windows, which depends on the window size specified in the classification model settings.

• Query Parameters:

In this pane the user can set parameters for different queries (Figure 10), specifically:

Figure 10.

Query Parameter pane, offering options for efficient data analysis.

• the time period to be analysed
• the species to be queried; here it is possible to filter for all species with a database index
• minimum and maximum of the confidence value; only the classifications within this interval are considered
• bin width: the size of the bins of the generated histogram
• the four buttons highlighted blue enable the following queries in the database:

• “Calculate bin sizes” produces a histogram for all predictions within the selected period and confidence intervals (cf. Figure 24).
• “Get daily histograms” creates histograms for each day within the selected time interval. Only bins within the selected confidence interval are considered (cf. Figure 23).
• “Get predictions” generates a CSV file with all predictions within the selected parameters.
• “Query” counts all prediction windows within the selected parameters and opens the Query Results pane.

• Query Results: In this pane, the user can draw a random selection from the query or retrieve the entire selection to perform a manual validation of the classifier results (Figure 11). Pressing the corresponding button extracts a zip archive from the original sound recording, which contains one audio file for each 5-second prediction window. The snippets include the corresponding time range of the prediction window as well as 5 seconds before and after the prediction window. In addition, an Excel file is created, containing the classifier-assigned confidence values, start and end time of the snippets in relation to the original recordings, as well as hyperlinks to the corresponding audio file. The user can set a high-pass filter, which is applied to each extracted recording if the checkbox is activated. The filter is double pole. The filter rolls off at 6 dB per pole per octave (20 dB per pole per decade).
• Jobs: Executed and currently running tasks are shown in the jobs section at the bottom of the query page (Figure 12). The user can download the results by clicking the download button.

Up to this date, 104 bird species in Britz and 54 in Melbgarten have been documented by sound recordings. For several other bird species, no reliably identifiable voucher recordings have been found yet (Britz = 21, Melbgarten = 6 species), mostly due to poor signal quality. However, we anticipate that audio signals from numerous other species could be found in the AMMOD soundscape recordings through systematic screening.

The species lists were complemented by means of classical observation. Systematic breeding bird surveys were carried out in 2021 (Melbgarten) and 2022 (Britz). In addition, unscheduled observations up to a distance of 5000 m from the AMMOD stations were taken into account. For Britz, reliable observations by third parties, some dating back several years, were considered in addition to project-related records. In the near vicinity of Melbgarten, i. e. up to a distance of 500 m from the recording site, observations took place in 2021 and 2022. For the wider area of the Melbgarten, reliable observations obtained from 2010 to 2022 were also considered.

In Britz, 49 bird species were detected by territory mapping of a 16-ha area with 7 visits. Together with the unscheduled observations, a total of 74 bird species were detected in the close vicinity of the recording station. Visual records of two additional species were reported by third parties. Only 11 of the species observed within a 500 m radius have not been found in sound recordings to date. Observations within a radius of up to 5000 m revealed 47 additional bird species. Of these, verifiable voucher recordings could not yet be found for 24 species. In total, 139 bird species could be reliably identified for Britz, 16 of them so far only in the sound recordings, but not by classical audiovisual observations.

In the Melbgarten area, 44 bird species were found by territory mapping of a 25-ha plot during 5 visits. Together with the unscheduled observations in 2021 and 2022, the number of species observed within a 500 m radius increases to 46, of which 10 species have not yet been found in sound recordings. Within a 5000 m radius, at least 68 additional bird species have been detected in the last 13 years, 53 of which have not yet been found in the sound recordings. Most of the additional bird species found through classical observation were migrants, roaming non-breeders, and species with territories outside the range of the AMMOD recording site. In total, 117 bird species could be reliably detected in the Melbgarten and its wider vicinity by the end of 2022, three of them so far only in the sound recordings but not by classical audiovisual observations.

Natural soundscapes are often complex. Four-channel recordings with high-quality directional microphones make it easier for the listener to distinguish between sound sources, since the signal amplitude is highest at the microphone pointing in the direction from which the sound originated (Figures 13, 14).

Figure 13.

Spectrogram of a four-channel recording. BRITZ02, 13 May 2022, 08:02 h; spectrogram settings – downsampling to 24 kHz, high-pass filter up to 950 Hz, DFT size 512 samples, grid spacing 46.9 Hz, Window Hann, overlap 50 %.

Figure 14.

Oscillogram of the recording presented in Figure 13. Since each microphone faces in another cardinal direction, the amplitude of audio signals differs considerably between channels.

The difference in signal amplitude between channels can be exploited for computing directional spectrograms (D-SPEC) in which colour represents direction of arrival (DOA) (Figure 15, top), see Baggenstoss et al. (2021) and Chapter 7 for details. Subsequently spatial processing is used to achieve grouping of nearby time-frequency bins into the same cluster (Figure 15, bottom). The directional clusters can be separated from each other (Figure 16, Table 4). Since in the directional clusters there are significantly fewer overlapping audio signals from different bird individuals and species, sounds emitted by specific individuals can be identified much more easily than in the original spectrogram (cf. Figure 13). However, some of the low-amplitude signals may be lost in the processing procedure.

Figure 15.

D-SPEC of the recording shown in Figures 13 and 14 before (top) and after spatial processing (bottom).

Figure 16.

Source separation into directional clusters, exemplarily shown for the recording presented in Figures 13 through 15 (Baggenstoss et al. 2021; Chapter 7 in this book). See Table 4 for details on the sound sources associated with each cluster.

The ultrasound sensor of AMMOD station BRITZ03 was deployed in a small forest glade. It was operated only between 14 and 26 July 2021, since hail destroyed the protective membrane of the microphone, which led to a device failure. A total of 686 minutes were recorded, of which 293 minutes contained bat activity found by manual screening (Figure 17).

Figure 17.

Spectrogram of an ultrasonic recording made with the microphone Dodotronic Ultramic 384. Besides echolocation and social calls of Leisler’s Bat Nyctalus leisleri there is a frequency band around 8 kHz, representing stridulation sounds of Great Green Bush-cricket Tettigonia viridissima. BRITZ03, 16 July 2021, 21:50 h; spectrogram settings – downsampling to 192 kHz, FFT size 1024, Frame size 50 %, Window Hann, overlap 87.5 %.

Another ultrasound device, BRITZ07, was deployed inside the forest, starting on 1 September 2022. In total 610 minutes of ultrasound recordings were revised manually, of which only 54 minutes contained bat activity. In general, the signals were weaker than those recorded at station BRITZ03 and more affected by reverberation. Figure 18 shows one of the better recordings.

Figure 18.

Spectrogram of an ultrasonic recording made with a Pettersson M500-384 mounted in a funnel, showing echolocation calls of Common Noctule Nyctalus noctula. BRITZ07, 7 September 2022, 21:50 h; spectrogram settings as in Figure 17.

Manually annotated recordings were used for the performance assessment of the classification model BirdID-Europe254. In total 19,844 audio signals were labelled in 99 soundscape recordings, most of which were collected at the AMMOD study sites Britz and Melbgarten. Since it was not possible to reliably identify all the signals to species level, in the end 14,683 annotated signals were available for performance tests (Table 5).

All visible and audible signals were labelled with bounding boxes in the spectrograms, even very faint sounds (Figure 19). The boxes also included the harmonics and reverberations of the signals (Figure 20). Besides species identity and identification level, additional tags were used, e. g. for vocalisation type. Since corresponding labels were not available in the training dataset used for the development of BirdID-Europe254, the latter annotations were ignored in the present work.

Figure 19.

A fully annotated stereo recording used for performance assessments. The bounding box #638, highlighted in red in the lower spectrogram, does not refer to a labelled audio signal, but to the segment shown in Figure 20. BRITZ01, 19 May 2019, 08:45 h.

Figure 20.

Detail of the spectrogram area highlighted red in Figure 19. Two 5-second segments are shown to illustrate the length of BirdID-Europe254 classification windows in relation to common bird songs and calls. Since in practice the analysis is computed with a step size of two seconds, we have omitted the respective overlapping classification windows for the sake of simplicity. Spectrogram settings – sample rate 48 kHz, DFT size 1024 samples, grid spacing 46.9 Hz, Window Hann, overlap 50 %. The spectrogram was truncated above 14 kHz because only noise was present in the high-frequency range.

Robust monitoring of bird vocalisations requires reliable species records, making high precision of classification results crucial. However, there always is a trade-off between precision and recall rate (Figures 21, 22). In other words, a higher threshold for confidence may lead to many correct-positive classifications excluded from analysis, while a lower threshold may result in unacceptably high levels of false-positive classifications. Regarding the performance of BirdID-Europe254 on AMMOD validation data (Figure 21), for some species, such as Mistle Thrush Turdus viscivorus and Marsh Tit Poecile palustris, performance dropped very fast when using a lower threshold for confidence, while in some other species, namely Great Tit Parus major, precision first increased before dropping off again. Only for some species it seems possible to find an acceptable compromise between precision and recall, e. g. Common Redstart Phoenicurus phoenicurus, Common Chaffinch Fringilla coelebs, Wood Warbler Phylloscopus sibilatrix, and European Robin Erithacus rubecula. However, since in the AMMOD validation dataset the number and quality of labelled audio signals differs considerably between species, there is the suspicion that at least some performance results may be biased (see Discussion). Not surprisingly, BirdID-Europe254 performance results look much better when validation is performed mainly with strong and non-overlapping audio signals typical of Xeno-Canto derived datasets (Figure 22). High average precision values are then achieved for most species. The strikingly low values for Eurasian Blackcap Sylvia atricapilla, Eurasian Siskin Spinus spinus, and Wood Warbler P. sibilatrix require further analysis. However, we suspect that these are artifacts due to, for instance, small sample sizes or a data processing problem, rather than true weaknesses in the classification models.

Figure 21.

Precision-recall curves for 24 representative species based on AMMOD validation data, computed for the classification model BirdID-Europe254. The average precision (AP) stated in the legend facilitates the correct assignment of similarly coloured curves to the correct species.

Figure 22.

Precision-recall curves for 24 representative species based on Xeno-Canto validation data, computed for the classification model BirdID-Europe254.

The raw classification results are the basis for all further analysis with respect to robust acoustic monitoring. In the first step, we made example plots for selected species, showing the distribution of classifications per day and confidence level (Figure 23). Here we chose the example of Common Redstart Phoenicurus phoenicurus, a long-distance migrant bird. In Germany, breeding populations usually arrive in April, though a few birds are sometimes detected already at the end of March (Südbeck et al. 2005). Considering this background, the onset of singing activity occurred on 15 April 2019. Assuming that the species was mostly absent before that date, almost all classifications in the first half of April 2019 potentially represent false-positive classifications.

Figure 23.

Raw classifications for Common Redstart Phoenicurus phoenicurus at BRITZ01 in April 2019 plotted by date and confidence level. The onset of vocal activity of the local breeding population on 15 April 2019 is clearly visible.

In the next step, we plotted the bin-size distribution for selected species, i. e. the number of classifications over bins of confidence values, exemplarily shown here for Common Redstart (Figure 24A–D). An exponential increase in bin size for the lower part of the confidence interval is observed at both sites considered, i. e. Britz (BRITZ01) and Melbgarten (MGB01) (Figure 24A, C). This pattern is probably due to the many classification windows in which the target species is not present (false positives). However, when zooming in on bins of higher confidence, there is a striking difference in patterns between sites. At BRITZ01 the increase in bin size towards higher confidence values is a strong indication for the species being present and abundant, while at MGB01 the species seems to be almost completely absent, as confirmed by manual validation and field surveys (Figure 24B, D). That is, at Melbgarten almost all classifications obtained for Common Redstart represent false positives, even those with rather high prediction probabilities.

Figure 24.

(A–D) Bin-size distribution of raw classifications for Common Redstart Phoenicurus phoenicurus at Britz (BRITZ01) (A, B) and Melbgarten (MGB01) (C, D) in spring 2022 (1 March to 30 June). X-axis: confidence level with a resolution of 0.02 (2 % steps), y-axis: number of classifications. Top (A, C): Bin-size distribution when the entire confidence interval is considered (≤ 0.02 to 1.0). Total number of classifications ‒ BRITZ01, n = 1,639,352; MGB01, n = 1,671,813. Bottom (B, D): Zoom-in on bins with confidence levels ≥ 0.2.

Team composition is crucial for the success of projects aiming at the development of hardware and software tools for PAM. Since acoustic monitoring of biodiversity concerns diverse aspects of life sciences, engineering, and logistics, multidisciplinary teams need to closely cooperate for data acquisition, device maintenance, software development, database management, performance assessment of classification models as well as analysis, interpretation, and validation of classification results. For instance, only due to the permanent interchange of ideas among computer engineers and bioacousticians it was possible to develop the web interface for efficient analysis of raw classification results presented here.

Custom-built hardware has some advantages over standard solutions. For example, it provides flexibility regarding the usage of high-quality directional microphones and wireless data transmission. Additional functionality is easily implemented, such as remote-control options and on-site data processing. However, long-term PAM studies must also consider other important aspects such as power supply, protection against vandalism, and the cost of purchasing and maintaining equipment and software. Hence, there are also many arguments in favour of using commercial audio recorders. If standard equipment is chosen, we strongly recommend energy-efficient stereo recorders with weatherproof electret microphones, for two reasons: Compared to mono recordings, stereo recordings provide some spatial information that facilitates data processing in many ways, whether for manual or computer-aided analysis. Equally important, electret microphones produce less self-noise and deliver much better audio quality than MEMS microphones, increasing the range monitored by each sensor unit.

In bioacoustic research, microphone arrays have been used to localise and track animals in the field (Rhinehart et al. 2020). Recently it was demonstrated that even an array consisting of only four directional microphones supports the separation of animal sound signals that overlap in time but originate from different directions (Baggenstoss et al. 2021) (cf. Figures 13–16, Table 4). Consequently, individual songs and calls can be reliably assigned to a sequence, even if they are superimposed by other sounds. Although further studies are needed for the wide practical application of this method, it clearly has an enormous potential for research on biodiversity, ecology, and animal behaviour. For example, the directional spectrograms may inspire the development of tools for signal annotation as well as for automated identification of species and single individuals. However, a crucial prerequisite for the widespread use would be the availability of weatherproof directional microphones, making sound-wave-deflecting constructions for rain protection superfluous (Baggenstoss et al. 2021; cf. Figure 1). For technical reasons, the development of weatherproof directional microphones is very challenging and thus requires the involvement of hardware engineers with experience in microphone design and acoustics.

A critical bottleneck for the development of robust pattern recognition models has been the availability of training and validation data (Jahn et al. 2017; Morfi et al. 2019). Soundscape recordings contain many acoustic signals that first need to be labelled, identified, and annotated (Figures 19, 20). This is a tedious task and requires expert knowledge and time. As resources are rare, the principal strategy for obtaining labelled audio recordings has been the extraction of audio files from public citizen-science and institutional animal sound collections. For the development of BirdID-Europe254 about 72,000 sound recordings for 254 European bird species were compiled from different sources, including the Animal Sound Archive at the Museum für Naturkunde (https://www.animalsoundarchive.org) and Xeno-Canto (www.xeno-canto.org). However, most of these recordings have only simple annotations, i. e. the identification of the principal species or species lists at best. Many of these recordings contain additional sound signals of competing species and other sound sources that are not annotated. This is problematic, as incomplete annotation of the training data can lead to a significant reduction in recognition performance. Therefore, considering that tens of thousands of manually annotated sound recordings may be required to develop successful recognition models (Kwok 2019), it is advisable to assign more than one bioacoustician to this task, especially if several animal groups such as birds, mammals, and insects are to be studied as part of a single project.

We emphasise that even if fully annotated validation files are available, this does not guarantee objective performance evaluation of classification models. The output may be biassed by the effect of long-tailed distribution of audio signals in soundscapes (Horn and Perona 2017). The reason is that few sound classes have many annotations and many classes are represented by only a few examples, particularly rare species and uncommon vocalisation types. Similarly, uneven distribution of sound quality can affect performance assessment results. A detailed analysis of BirdID-Europe254 precision-recall curves for AMMOD validation data revealed that at least some of the strange-looking curves are the result of such biases and do not reflect the actual performance of the classification models (Figure 21). For example, the rapid decrease of precision for the Mistle Thrush Turdus viscivorus is caused by the long-tailed distribution regarding signal quality, with many low-amplitude song phrases and very few high-amplitude signals annotated. In other species, strange effects are caused when the validation dataset accidentally contains a higher proportion of rare call and song types than the training dataset. This was likely the case for the Great Tit Parus major for which precision first increased before dropping off again. For both species the precision-recall curves look very differently when assessed with Xeno-Canto data (Figure 22), where recordings with high-amplitude signals of commonly heard vocalisation types outnumber recordings of poor quality and rarely heard call and song types. Conclusively, it is not enough to assess classification results only on the basis of allegedly objective validation datasets. Instead, representative random samples of classification results should be drawn to manually determine precision for each target species, recording station, and monitoring period.

In multilabel classification, each classification window is assigned a confidence value for the presence of each species included in the model. This means that for a given number of audio recordings, the number of classifications is the same for all species. Because classification windows in which a particular species is highly unlikely to occur receive much lower confidence values than those in which the species is likely to occur, the bin-size distribution typically takes the form of an L lying on its long side (Figure 24A, C). Consequently, the short side of the L contains almost exclusively false-positive classifications, while the long side may also contain correct-positive classifications. To get a better idea of the number of classification windows that may contain correct-positive results, it is necessary to zoom in on the range of higher confidence within the bin-size plots (Figure 24B, D). For Common Redstart Phoenicurus phoenicurus, the difference in bin-size distribution between sites where the species regularly occurs (e. g. BRITZ01) and those where it does not (MGB01) is striking. Therefore, based on bin-size plots we anticipate the development of a tool for automated analysis of the presence/absence status of species. To this end, several aspects should be considered:

1. Even if a species is completely absent, some classification windows with quite high confidence values still appear in the plots (Figure 24D). Thus, if it is important to know whether a species is rare or not present at all, it is always necessary to manually check whether there are some true positives among the classifications with high confidence values.
2. Likewise, for species that are present with certainty, it is important to draw representative random samples for determining the confidence threshold, above which almost all classifications can be considered as correct positives.
3. In many cases, the situation may not be as obvious as in the example of the Common Redstart. For instance, if a species is rare at a recording station or occurs only at the edge of its detection range, the increase in bin size for bins with high confidence values is much smaller than for a species that is common in the closer vicinity of the recording station. These and other factors, such as the presence of competing species whose vocalisations are similar to those of the target species, can result in complex bin-size distribution patterns that can only be interpreted based on manual validation of random samples.
4. Because acoustic conditions vary from site to site and at any given site also vary from year to year, manual validation must be repeated periodically for all target species at all sites. This is currently the only way to obtain robust data on species acoustic activity that is comparable between sites and time periods.

The following are the key parameters that affect the calculation of D-DSPEC.

Parameter description: Variable name:

Number of microphones. M, nmic

FFT size (window size in samples). N

processing window shift (samples). shft = N/3 (i. e. 2/3 overlap)

angular resolution (for beam-forming). ang_res (degrees)

speed of sound. c (m/s)

sampling rate. fs (samples per second)

microphone separation. d (m)

break frequency for high-pass filter. fmin (Hz)

start and end times for processing. (tstart, tend) (seconds from start of recording)

number of angles (beams). na

Data is read in for a specified time range (tstart,tend) from the .wav files, which contains 4-channel data. The function read_wave.read_wav() is reads just one channel (microphone) at a time from the specified time range. This time-series is passed to the spectrogram function that calculates a spectrogram with FFT size N and overlap 2/3. The FFT size N must be divisible by 3. High-pass filtering is done by multiplying by the filter frequency response, which is stored in the variable frsp. Some scaling is done so that the Python code has the same values as the MATLAB/OCTAVE code. The spectrogram data for channel ch is stored in the list variable B[ch].

The code to do beamforming and calculating the D-Spec is provided in the function ammod_utils.get_cspect(). Memory for the D-Spec output is allocated using the variable I, which has shape (n,NSEG,3), where n is the number of FFT bins and NSEG is the total number of spectrogram time segments (called nseg elsewhere). The third dimension is the RGB color space.

The code loops over the NSEG segments, processing one segment at a time. For a given time step „ismp“, the following steps are made:

Relevant parameters are listed as follows, along with default values.

Parameter: Default value: Description:

expfac # .5 # (make larger to give more weight to higher signal strength)

lfac # 1.0 # (make larger to give more weight to color (direction))

minwt # 1e-3 # mixture weight assigned to noise

radius # 6.0 # search radius for neighborhoods (pixels)

dfac # 30.0 # Make larger to give more weight to spatial effects

P # 2 # GMM mixture components per cluster

nit_gmm # 90 # number of GMM iterations to execute

nit_spatial # 10 # number of spatial iterations to execute

minvar # .00015 # Minimum GMM variance. Increase for fewer clusters

merge_thresh # -.2 # Threshold to merge clusters. Decrease for fewer clusters

nhbd_type # 2. # 1 = r^2/d^2 2 = exp (- d^2/r^2)

The following steps are required prior to clustering iterations.

The clustering algorithm has two phases. The first phase is essentially a Gaussian mixture model estimation algorithm, which is a type of E-M algorithm. The GMM is applied to 1-dimensional features (just hue), so is unable to apply spatial information. Without spatial information, the pixels assigned to a cluster can be distributed randomly across the spectrogram. A second spatial processing phase is applied once the GMM has converged. In the spatial phase, we take a look at the pixels around a given pixel. The cluster that these neigboring pixels are assigned to will then influence the assignment of the given pixel.

To use a recording for the D-SPEC software, it needs to be 4-channel in .wav format. Optimal sampling rate is 24 Khz. If the file is in 48 kHz sampling rate, a utility program is provided to convert it to 24 kHz. Using MATLAB or OCTAVE, invoke the program dsamp2.m:

>> dsamp2(‘file1.wav’,’file1_ds2.wav’)

This will down-sample the file file1.wav and save it as file1_ds2.wav. To enter the file into the file list, open the file file_list.txt in an editor, we enter a text line with 7 fields. Using the example of file1_ds2.wav), the line looks like

File46 24000 0.27 343 4 “file1_ds2.wav” “None”

The first entry is a ‘nickname’, a short string to identify the file. The convention is to use FileXX, and make XX a number that has not already been assigned. The second entry is the file sampling rate. The third entry is the array microphone separation in meters. The fourth is the speed of sound in m/s when the recording was made. The fifth entry is the number of microphones. The sixth entry is the file name in quotes. The last entry is the name of an annotation file in quotes, or ‘None’ if there is none.

The D-SPEC software is launched simply by invoking python on the program ammod_gui.py. In Linux:

$ python ammod_gui.py

Launching the software from Windows might differ slightly.

The graphical interface comes up with default parameter settings. These can be changed by the operator, and saved in a parameter file by specifying the file name and pressing SAVE on the top of the window, or LOAD to retrieve the saved parameters.

To load a data set, use the File: drop-down manu to select the file. On the left of the drop-down menu appears the nick-name that was entered in the file list. Once this has been selected, the information stored in the file list (microphone separation d, speed of sound c, sample rate fs, number of microphones nmic) is transferred to the parameters displayed in the GUI.

Before computing the D-SPEC, the operator can specify a desired time range (T1, T2). The total time processed at a time can be up to 30 seconds, although the time to process and cluster increases with time range. The time window can be moved forward or backwards by 2 seconds using the (+2, -2) keys.

Once the time range has been set, press the CSPEC button to compute the D-SPEC. The D-SPEC will not be displayed until the requested by the PLT button. The brightness (brt) and contrast (contr) can be changed to optimize the display, then only the PLT button needs to be pressed. Note that these values will be saved or loaded when using the SAVE/LOAD buttons. Note: never ‘kill’ the D-SPEC graphics window. This will cause errors.

Many of the parameters that affect D-SPEC are pre-determined by the array and recording, but the following can be changed: N, ang_res, fmin, T1, T2 Some details of these are listed in Section 7.2.1. The FFT size N can be experimentally changed. Generally, for a 24000 Hz sample rate, 384 is a good choice, and this would change proportional to sample rate. The highpass-filter cuttoff ‘fmin’ should be set experimentally for each recording to eliminate low-frequency noise.

Once the D-SPEC has been computed, the clustering can be started by pressing ‘Cluster’. Information is displayed in the system console. Once the clustering operation ahs stopped, the cluster-specific spectrograms can be displayed using the ‘PLOT Clusters’ button.

Operator-settable parameters include the following:

thr, rel_thr Threshold. When the rel_thr box is checked, then thr defines the fraction of D-SPEC pixels that are going to exceed the threshold. Default is 0.1 for keeping 10 % of the pixels. When the rel_thr box is not checked, then thr defines the actual amplitude threshold. A good starting value is 0.3.

expfac. Exponential factor. Default value is 0.5. There is little need to change this.

nclus. Default value of 9. This is the starting number of clusters. There is little need to change this.

P. Number of GMM components per cluster. P=2 is default and there is little need to change this.

lfac. This is the factor multiplying the GMM log-likelihood. This is experimental, and there is no need to change this.

minwt. Minimum weight, with a default value of 0.001. It is used for the ‘dummy’ cluster, a cluster that is just there to gather un-assigned pixels. This is experimental, and there is no need to change this.

radius. This defines the neighborhood radius for spatial processing. Larger neighborhood should promote smoother-looking results. Default is 6 pixels. There is no need to change this.

dfac. This determines the amount of spatial influence to the clustering. Default is 30. Higher will result in clustering that will produce more local consistency.

nit_gmm, nit_spatial. Number of GMM and spatial iterations. Default is 40 and 5. Making nit_gmm smaller makes tthe clustering run faster, but if time is not an issue, use 100 iterations. nit_spatial should be changed with care. Too many spatial iterations might result in poor results.

wts_exp. Default is 0.01. Making this higher will promote faster convergence. This is experimental, and there is no need to change this.

minvar. This parameters is important in determining the final number of clusters. Default is .00015. If two signals are coming from nearby angles, they will produce nearby clusters in the hue space. Minvar defines the minimum GMM variance. By making this large, the GMM cannot resolve two closely lying clusters in the hue space, so is likely to group them into one cluster. However, making this too small can result in a single signal being split into multiple clusters.

merge_thr. This is the threshold for cluster merging, a negative number. If the log-distance between clusters is above this, they will get merged. To promote fewer clusters, make this lower (more negative).

nplot. Defines the number of sub-plots when plotting clusters, so is effectively the largest number of clusters to display.

To generate cluster-specific time-series, press the ‘Gen Timeseries’ button. This causes the files ‘cluster1.wav’, ‘cluster2.wav’, etc. to be written. These files can be used for external purposes, such as classifying. Or, they can be played by selecting the desired cluster ‘CLUS’ and pressing ‘Play’.

NN-DSPEC processing is the same as D-SPEC up to the formation of beams. Instead of forming 120 beams in 3-degree increments, NN-DSPEC operates by creating 32 pre-formed beams in 11.25 degree increments, but is otherwise the same as laid out in Section 2.4. The amplitude of these beams is computed, resulting in a beam output with dimension (nseg,n,32). This is a spectrogram with 32-dimensional bins. This 32-dimensional feature is passed to a neural network to convert the 32-dimensional bind to 3-dimensional (RGB) bins. From the perspective of the neural network, it can be seen as just a 32-dimensional feature with nseg*n samples. The neural network itself is very simple. There are 5 network layers, each has 32 neurons except for the last that has 3 neurons. Each layer has the soft-plus activation function. The network is simply trained to re-create the ground-truth which is an artificial RGB D-SPEC.

Since the implementation is spread across mutiple experimental programs, we cannot give a complete description here. Instead, we point to the required programs and their purpose, so that the process can be repeated in the future.

We first address the stereo camera design (cameras and Baseline, design goals 1 and 3). The raw data produced by the cameras is processed and stored by the control unit (design goals 2 and 3). Weather-resistance (design goal 4) is provided by the case. Infrared motion Detection and illumination facilitate energy efficiency (design goal 2) and operability at night time (design goal 1a). We additionally describe in detail the power supply, how we obtain animal-camera distances using stereo correspondence and how the captured data may be transferred using different connectivity options. Table 1 summarizes the bill of materials.

Cameras and Baseline: A pair of Raspberry Pi High Quality Cameras were chosen for their cost-effectiveness and the high sensitivity of their Sony IMX477 sensor (Sony Semiconductor Solutions Corporation, no date). Interchangeable lenses allow adaptation to specific scenarios (i. e. shorter focal lengths for close-up scenes, higher focal lengths for more distant objects). Removal of the infrared filter allows sufficient exposure at night using artificial infrared illumination. The cameras have an additional Bayer filter above the sensor, which is usually responsible for filtering different wavelengths to create a color image. We leave this filter intact to not risk damaging the sensor itself. As near-infrared illumination (either from the environment or the illuminator) illuminates all color bands, we do not try to recover any color information and instead average all bands to obtain a grayscale image. The cameras are mounted on a long U-shaped aluminum rail with holes drilled at regular intervals to allow configuration of different baseline distances between both cameras. Both cameras are connected through long ribbon cables to the two MIPI CSI-2 interfaces of an NVIDIA Jetson Nano Developer Kit. Both design aspects, i. e., the interchangeable high quality lenses as well as the configurable baseline construction, allow for adaptation to specific scenarios, namely free fields, feeding places, animal crosses, green bridges, etc., where animals are observable at different distances.

Control: We use an NVIDIA Jetson Nano Developer Kit as the central control and storage unit. It is responsible for taking motion detection signals from the PIR sensor, turning on the power to the IR illuminator, capturing, encoding, and archiving image material from the cameras. We decided on the Jetson Nano for the following reasons: (1) compared to most single-board computers, it provides two MIPI CSI-2 interfaces for the two cameras, (2) it provides a powerful GPU that can be used for encoding video efficiently, and (3) it supports a power-efficient hibernation mode. The Jetson Nano uses a 128GB microSDXC card for persistent storage.

Case: To make SOCRATES as weather-resistant as possible, most components are placed inside a single weather-proof case. The case is made of 0.8 cm thick birch plywood and is 80 cm wide, 11.6 cm high and 20 cm deep. We decided for a very wide case to be able to adapt the baseline of the stereo camera to different configurations. The front of the case is shielded by a piece of acrylic glass. In the bottom, we add a 4 cm wide, circular hole for ventilation, which is covered from the inside with an insect screen. The battery is mounted via Velcro strip onto a hatch in the bottom of the case, to allow quick replacement. We add two further holes for the wiring of the IR illuminator and motion detector, respectively, both of which are sealed using silicone. The top of the case is sealed using a silicone strip and secured by screws, which can be loosened to take it off for maintenance. All exposed wooden parts are further treated with marine varnish for weather resistance.

Motion Detection: Like most camera traps, we use a pyroelectric infrared (PIR) sensor for detecting motion and thereby triggering capture. We choose an HC-SR501 PIR sensor due to its compatibility with the 3.3 V GPIO pins of the Jetson Nano. We initially mounted the PIR sensor inside the case, just behind the acrylic glass. However, we found that this severely impaired the ability of the sensor to detect any kind of motion outside the case. This is because acrylic glass is opaque around wavelengths of 10 μm (Altuglas International 2000), which corresponds to the body temperatures of most animals. Therefore, we mount the PIR sensor in a separate 3D printed weatherproof casing below the main case.

Illumination: We employ a simple 12 W, 850 nm infrared illuminator to ensure properly exposed images at night without disturbing most animals. The illuminator has a weatherproof case and is mounted on the bottom of the main case. The 12 V power supply is switched by a Jetson Nano GPIO pin using an IRLZ44NPBF MOSFET.

Power supply: All components are powered by a lithium ion polymer battery, which has a high power density. We employ a battery with a theoretical capacity of 236.8 Wh (1600 mAh at 14.8 V). A generic 4S balancer circuit board provides over-discharge protection. The variable voltage of the battery is then regulated to 5 V for the Jetson Nano and 12 V for the infrared illuminator by Mean Well SCW20A-05 and SCW12A-12 converters, respectively.

Connectivity: SOCRATES may transmit the recorded data via three different means: wired ethernet cable, wireless LAN (Edimax EW-7811UN) or cellular connection (Huawei E3372H). If no basestation is available, we use the cellular connection to manually download the captured data. Otherwise, we connect via wireless LAN and the CoAP protocol (Bormann, Castellani, and Shelby 2012) to the AMMOD Basestation (Wägele et al. 2022; Sixdenier et al. 2022), which in turn uploads the captured data to the AMMOD Portal (cf. section 8.4).

A significant drawback of the custom SOCRATES hardware is the difficulty of producing a high volume of units, which is primarily due to the manually assembled case. Compared to commercially available monocular camera traps, mean power consumption is relatively high at around 1W, which necessitates relatively short battery exchange intervals of around 8 days. Finally, if possible, the wavelength of the infrared illuminator should be increased to around the widespread 940 nm. We did not notice any adverse reaction of observed individuals to the current 850 nm illumination, however, it might scare some other species with wider spectral sensitivity.

The control software of SOCRATES is responsible for reacting to signals from the PIR motion detector (cf. section 8.2.1), triggering capture, and communicating with the AMMOD basestation (cf. section 8.4). It is implemented in Python and runs on Linux via the NVIDIA JetPack SDK. We furthermore adjust the device tree to allow the PIR to trigger a wake up signal via the GPIO16 pin. This allows the control software to put the SOC into the power-efficient SC7 mode during periods of inactivity and resume once motion is detected. Once motion is detected, video material is encoded on the Jetson Nano’s GPU by synchronizing the image streams from the left and right cameras, concatenating them horizontally, and compressing the resulting video of resolution 2 × 1920 × 1080 using the HEVC video codec (Sullivan et al. 2012). This results in a bitrate of roughly 6.7 Mbit/s.

The central goal of SOCRATES is to infer depth information through stereo vision. In the natural world, as well as in computer vision, this is achieved by solving the stereo correspondence problem. To solve the stereo correspondence problem efficiently, the left and right images must be rectified. To obtain an accurate rectification, the intrinsic (internal camera parameters) and extrinsic (rotation and translation between the cameras) parameters have to be obtained by a calibration procedure. For the calibration of the intrinsic parameters, a calibration object (e. g. checkerboard pattern printed on cardboard) has to be captured by the camera(s) to be able to associate 3D points in the scene with 2D points in the resulting image. To obtain the extrinsic parameters, eight or more correspondences between images of points in the projections of both cameras must be established (Longuet-Higgins 1981). We perform both intrinsic and extrinsic calibration using Kalibr (Maye et al. 2013) with a grid of AprilTags (Olson 2011) mounted on a wooden board as the calibration target. During the setup of SOCRATES, the calibration target is manually moved through the scene such that it covers as much as possible of each camera’s field of view. After SOCRATES is assembled and calibrated, calibration does not have to be repeated when deployed to different locations, as the calibration is not dependend not on a specific location but only on the camera configuration. Given the intrinsic and extrinsic parameters, we rectify the images of both cameras and compute the disparity of each pixel using using Li et al. (2022).

To detect and localize animals in the images produced by SOCRATES, we use the mmdetection (Chen et al. 2019) implementation of the Cascade Mask R-CNN (Cai and Vasconcelos 2018) instance segmentation model. To take advantage of the additional depth information obtained via stereo correspondence (cf. section 8.3.2), we fine-tune an Omnivore-L (Girdhar et al. 2022) backbone, which is pre-trained on both color and depth images. Compared to a color-only baseline, incorporation of the depth information yields a substantial improvement in detection accuracy (e. g. by in bounding box; Haucke et al. 2022). The resulting bounding boxes and instance masks may then be combined with the depth images (cf. section 8.5) to sample per-animal distance measurements which can be used for estimating abundance of unmarked animal populations (Haucke et al. 2022).

The software context is divided into modules, each running on its own thread, which can belong to three classes:

• An uploader, which retrieves files either from the buffer in memory or from the storage and uploads them to the AMMOD portal.
• A collector, which collects data from sensors, either asynchronously by waiting for a sensor notification or by asking the sensor periodically for data (i. e. polling). Each collector collects files from one sensor and can use a different method for it. Collectors are either spawned at startup or every time a new sensor exposing a SensorAPI interface is discovered.
• A master which is in charge of managing all the modules, including implementing the energy management policy.

As of the current time of writing, all sensors are connected to an Ethernet network (except for the sensing functions in the power supply, as already mentioned and the weather station, which is directly connected via USB). To unify their interfacing, it has been decided to write a library that handles the transmission to the base station, this library is referred to throughout this chapter as SensorAPI. The library uses the Constrained Application Protocol (CoAP), which is more lightweight, but similar, to HTTP, to transmit the given files to the base station.

The resource discovery feature is used for the automated discovery of the sensors, which means they can be added and removed from a site without touching the configuration file. The observation feature allows for sensors to trigger a GET request from the base station when a new file is produced. This is more energy-efficient than active waiting, and its embedding in the library is more convenient for a developer integrating a sensor.

The library is written in C++ and a Python binding is provided. It is available on a publicly accessible Gitlab repository¹, alongside more in-depth documentation.

The base station software is configured via a JSON file that follows a predefined schema. The configuration has three main properties:

1. Description of the sensors it is connected to, except those connected via the SensorAPI (because they are not discovered automatically), e. g. the weather station.
2. The settings of the base station: Whether to check the md5-checksum of the transmitted files, if the discovery of sensors via SensorAPI should be enabled, the period of uploading, and so on.
3. The characteristic of the deployment site: Its coordinates, the device identifier of the base station given by the AMMOD portal, and its serial number.

An exhaustive description of the configuration file is available in the git repository.

To access the base station from the Internet for maintenance purposes, we deployed a VPN running on a server accessible from the outside. For security reasons, it is possible to open an SSH connection to a base station only when inside this VPN.

The VPN server is currently hosted on a computer on the FAU campus. In the production phase, it could be hosted alongside the other AMMOD services.

From a research-oriented perspective, we tackled the problem of designing a base station (e. g. finding the right battery size, the right PV module, and an optimal energy management policy) by leveraging Design Space Exploration techniques. An overview of this process is shown in Figure 4.

Figure 4.

An overview of SIDAM.

An analytical parametric model of a typical AMMOD deployment site (a base station and its sensors) was designed and run inside a custom simulator. The parameters of this model are divided into a site-specific part (e. g. number of sensors, their type, their power consumption, etc.) and a base station part (e. g. the battery size, the size and number of hard drives, etc.) At the end of a simulation, the simulator outputs some metrics of the system: uptime of the base station, total amount of data transmitted, etc. These metrics, alongside the financial cost of the components, can be used as objectives for a Multi-Objective Evolutionary Algorithm (MOEA), which optimises the base station parameters.

Our results show that it can be financially interesting to have different base stations that have a different cost range to be deployed in different locations. For example, a base station deployed in a sunny region of Germany requires cheaper components than one deployed in a rainy region. We also use our tool to evaluate different energy management policies and to optimise the parameters of such policies. This methodology, which is named SIDAM, and its results were published in Sixdenier et al. (2022).

All the data collected by the sensors and send to the base station is prepro­cessed before it is forwarded on to the AMMOD cloud. Erroneous data and false positives are discarded and data is compressed. Depending on the configuration of the base station and the activity of the sensors still many gigabytes of useful data may be generated each day. To get any generated files to the AMMOD cloud where it can be stored for the long-term, fully analysed and presented to the public.

To transfer this amount of data different technologies can be utilised. Wired solutions like Ethernet or DSL are the most efficient approach but usually not available as the AMMOD stations need to be placed where animals roam freely, not where the man-made infrastructure is best.

The wireless technology that provides the best compromise between data rates, power consumption, availability of parts and long range network coverage within Germany is 4G/LTE. Any Internet of Things (IoT) focused long range technology like LoRa usually does not allow for a high enough data throughput. Satellite communication links are an interesting option to look at for very remote locations where 4G/LTE coverage is insufficient. Those however usually require more expensive hardware, a more complex setup and have a higher power consumption. For the default AMMOD station these are too expensive.

Another alternative is the implementation of a directional radio link between the AMMOD base station and either an 4G base station or a 4G client specifically set up for the usage in AMMOD in a location with better cell reception, possibly via multiple hops. This is always a more expensive setup than just implementing a 4G modem and antenna in the AMMOD base station itself but can be very valuable for the setup in for example a remote valley. The directional link itself can be more efficient in transmitting the sending power to to next communication partner, but that is only an advantage if that device does not need to supply a 4G modem with battery power as well.

The huge advantage 4G/LTE has over most any competitive technology at the moment of writing is the availability of systems. A 4G modem can be bought relatively cheaply and be effectively enhanced with a directional antenna on a mast a few meters high. Such an additional antenna allows for a very efficient transmission of signals and therefore the required sending power can be greatly reduced. The mast assures, that the antenna is as close as possible to having a direct line of sight to the nearest 4G base station. Such a static setup prevents any mobility of the system but this is in line with the requirements for the stationary AMMOD base station. A precise orientation of the antenna needs only to be done once when the station is assembled first.

For the test sites that where set up within the AMMOD Project, a few different configurations where examined. The following is the configuration of the ones used at the current test sites. Different hardware can be used in a similar manner to achieve comparable results.

In this example implementation a RUT240 Industrial Cellular Router by Teltonika Networks is used for the 4G/LTE radio link. It is specifically designed for Machine to Machine Communication, provides good interfaces which can be accessed and modified easily externally and a low power consumption. The modem accepts a comparatively wide range of DC voltages which allows it to be used with either a 12 V or a 24 V supply voltage from the base stations power lines. The two external antennas for MIMO-4G communication were exchanged for a WB 23 antenna by Wittenberg, which is shown in Figure 7. This allows for a very directional and efficient radio link. Combined with a 2 m mast the test sites can be provided with an more than adequate internet connection.

For the test site in Bonn, a mast-setup was used that has minimal impact on the setup area and is thus suitible to be set up in natural reserves. Four ground anchors are used in conjunction with one steel wire each to keep the mast upright. This setup is depicted in Figure 6.

Figure 6.

The setup of a minimal-impact antenna mast.

Figure 7.

An view on the used LTE antenna (white) and the weather station on its extension arm.

The sensor links allowing for data transfer between the sensors and the base station need to be very flexible as they are required to work for many different scenarios. The positioning of the sensors is determined through its function. Visual sensors need to have a clear view onto places of interest, for example an animal trail or a den. Audio sensors work best if they are shielded from unwanted noise, and insect traps will be placed where insects are commonly found. Sensors may be placed directly at the base station or 100 m away. There might also be a free line of sight or many obstacles between base station and sensor. Therefore the path between the sensors and the base station will usually be suboptimal for the transfer of data. To provide these data links, we investigated different technologies.

As with the internet link, a wired connection will always provide the best reliability in data transfer and the highest power efficiency compared to wireless solutions. This is always the preferred solution for sensors close or directly attached to the base station. An Ethernet connection is usually the most commonly available option and can be integrated into many systems easily. While it also allows for power transfer via Power over Ethernet, it does impose a certain amount of overhead and is not really adequate for minimal systems. A lower level wired link that is commonly used, especially for industrial applications, would be a serial interface like RS-232 of RS-485 over which protocols like Modbus can be transferred. This is a simpler and more efficient approach that does not require full internet capabilities from the sensor but is limited in its bandwidth when compared to the Ethernet standard. This is the preferred option when connecting a close sensor up top 10 m away from the base station, which does not generate a large data volume. When connecting a sensor that generates high data volumes of 1 GB per day or more, like video sensors, to the base station, a wired connection is usually the only option as any wireless solution with a high enough data rate consumes also a very high amount of data. In these cases, Ethernet will usually be chosen as such sensors are also fairly powerful in terms of computational capabilities and Ethernet does not provide much of a burden. Also any sensors that draw power from the base station need a wired power connection. Along the same routing, a data link cable can also be placed which makes a wired solution realisable with very little effort.

The most powerful wireless technology in terms of data rate, flexibility, and compatibility is WiFi. It does, however, also consume more power if compared to other technologies, which are more focused on an application within the Internet of Things (IoT). It also has a range limit of 100 m, which may not be enough for some sensors. The same problem of range limitation has Bluetooth which builds upon a lot of WiFi technologies and does not provide any real benefit for the AMMOD scenario. ZigBee also focuses on short range communication usually below 100 m. It is capable of communication over longer distances in good conditions and might be a good fit for specific AMMOD scenarios but is limited to low data rates and is not tested further in this AMMOD phase as there are alternatives that offer a better feature set. WiFi is the preferred option for sensors that generate considerable amounts of data between about 100 MB and 1 GB per day and need to be placed too far away from the base station to actually be connected with a cable.

Some sensors are spread out over a large area to collect data optimally. For example insect traps may be set up across a large field or around a wooded area. In a constellation like this, distances between 100 m and 1 km are common. As these traps actually collect biological samples and do not generate large amounts of data, a long-range, low throughput wireless link is ideal to realise an efficient link. An approach with multipath-propagation might also be interesting to research but is not within the scope of this project. LoRa is a long range protocol useful for Internet of Things (IoT) application which is suitable for the aforementioned scenario. It allows for communication over up to 10 km at data rates of up to 50 kbit per second. It is also widely available. The LoRa-WAN network is a large area network which covers wide parts of Germany. It provides an alternative access method to the internet for base stations or stand-alone sensors which produce only limited amounts of data. It covers mostly areas which are also covered but LTE but could be expanded upon for specific AMMOD-scenarios and is considerably cheaper to access than a contract with an LTE service provider. LoRa is the solution of choice for the AMMOD project to connect any sensor that is placed far away from the base station above 50 m.

For the LoRa connectivity of the sensors, different approaches are possible. If the sensor utilises a Raspberry Pi or an Arduino compatible system, there are extension boards available that integrate a LoRa modem on a low level. There are also USB solutions which can be utilised for the base station connectivity and can be an option for the sensors as well, if other methods of integration are not viable.

To analyse the large amount of data collected by all AMMOD systems and correlate this data correctly, a precise time synchronisation over all systems is necessary. That way all deployed sensors as well as all AMMOD base stations assign the correct time stamp to each measurement. The external timer used within the AMMOD project is derived from the Global Positioning System (GPS). LTE/GSM could also be used as a reference. Their coverage is however not as universal in very remote areas. A GPS receiver can also be integrated into every wireless sensor to achieve time synchronisation independent of the AMMOD base station. Sensors directly connected to the base station with a wire can have their clocks updated from the base station through NTP and do not need to invest in their own GPS receiver. The GPS modem also provides location data which can be used to map the layout of an AMMOD site automatically and possibly be useful in the recovery of lost or stolen equipment.

The time synchronisation can be used to connect two or more local measurements. For example if an animal is detected by a video camera and a microphone, movements and sounds can be correlated if they are registered at the same moment. Also data from multiple base station can combined. For example reactions to a lightning strike may be observed at multiple stations within a certain range. Also this synchronisation remediates clock drift. For example, events like a sunset can be reliably targeted with scheduled measurement runs.

Just like the time of day, or more practically the position of the sun, weather data is very useful to give context to animal and plant behaviour. Rain and wind for example can greatly influence the flight of insects and birds. Certain temperatures or values of air humidity may trigger the expulsion of pollen which can be detected. Also certain weather conditions may prevent reliable measurements by some types of sensors. Heavy movement of leaves through strong winds may falsely trigger motion sensors for video sensors. The noise of heavy rainfall may drown out any animal sounds detectable by audio sensors. Thus deactivating these sensors in adverse weather can save a lot of power and reduce the amount of unusable data collected.

Data drawn from weather services may also be an option for a minimal AMMOD station were it is not feasible to invest into a local weather station. A dedicated local solution is however preferable as it provides a far better spacial and chronological resolution. It should be emphasised that only a local station can actually measure local effects such as the wind direction at a forest boundary or the air temperature and humidity on a river bank.

A good weather station measures all the necessary weather data as well as GPS data needed for the operation of an AMMOD station. For the test sites of the AMMOD project, the Weather Station Compact WSC11 from Thies Clima was used. It is a compact system that was designed for local weather monitoring. It measures a wide range of values like temperature, humidity, air pressure, wind speed and direction, precipitation, and illumination. A GPS receiver is also integrated. The precision of the data provided and the measurement setup do not allow for a usage in a meteorological application but is more than enough for the kind of context data required.

The WSC11 can be mounted to the same mast as the antennas of the base station. Ideally the weather station is mounted to the very top as any components above it may obstruct its sensors for sun and rain detection. Many adaptors and extensions are available from the manufacturer to facilitate this. The WSC11 can give analogue outputs which can be digitised externally. It can also be controlled and read out via Modbus, which was implemented for the AMMOD project. The WSC11 was bought with the optional, ready-for-use 5-core cable through which the Modbus and power lines run. These can be split in the base station housing and connected to the base station’s power supply and a RS485-to-USB adaptor. The Modbus calls for the base station where implemented as a dedicated library. This way a modular integration into the base station as described in Section 9.3.2 was realised.

The requirements for connecting base stations to the AMMOD data portal are grouped as follows:

• Base station and sensor requirements
• Data requirements
• User requirements

10.2.1.1 Base station and sensor requirements

Base stations and sensors need to be registered and assigned a unique identifier (deviceID) before the integration with the AMMOD data portal. This is done using the AWI sensor registry, (which instance of the sensor registry is used for which instance of the AMMOD data portal is detailed in the introduction to this chapter). Upon filling up the registration form with all mandatory information of the base station or sensor, the deviceID is assigned by the sensor registry manager. The AMMOD data portal is automatically pulling the base stations and sensors-related information from the registry to visualize all the deployed AMMOD devices in a map and tabular view. Only uploads with valid deviceID are accepted. Figure 2 clarifies this aspects.

Figure 2.

Registration, assignment and management of sensor-related information.

It is highly recommended to nominate a sensor registry manager, who is responsible for the registration process and updating sensor’s metadata. The AMMOD base station operator/s is/are responsible for the sensor and data for one or more collection sites.

The sensor registry manager needs to create an account for the AWI sensor registry. This is currently done by sending the request via e-mail to o2a-support@awi.de. To get familiar with the functionalities of the service is recommended to visit the tutorial webpage of the AWI web service registry https://sensor.awi.de/?site=tutorial.

Please refer to Figure 3 and note the following for a correct integration of the base station during registration:

Figure 3.

Example of registered base station with assigned deviceID 7390.

• The field “Long Name” MUST start with the word “Ammod” (case insensitive)
• From the drop-down menu of the “Type” field, “station” MUST be selected.
• In the “Local Frame” page, the geographical coordinates in decimal degrees of the base station MUST be provided.
• The field “Collections” becomes available after the registration is finished. “AMMOD” MUST be selected among the available collections in the registry.
• The “State” field MUST be set to “Public” (default is “Construction”) to make it ready for the AMMOD data portal.
• The fields in the other pages of the form, e. g. Contacts, are not mandatory but nice to have, e. g. Data provider.

Please refer to Figure 3 and note the following for a correct integration of a sensor during registration:

• The field “Long Name” MUST start with the word “Ammod” (case insensitive).
• The serial number of the physical sensor MUST be provided in the field “Serial”.
• If no specific “Model” details are available then “not specified” MUST be set in the field.
• “Bioacoustic”, “Camera”, “Malaise trap”, “Pollen trap”, “Smellscape” and “Weather station or meteorological observation” are AMMOD-specific terms available in the drop-down list of the field “Type”.
• Once the sensor is registered it needs to be added as a “Subdevices” of the belonging base station.
• The “State” field MUST be set to “Public” (default is “Construction”) to make it ready for the AMMOD data portal.
• In the “Local Frame” page, the geographical coordinates of the sensor are only mandatory for standalone deployed sensors, i. e. not declared as subdevice of any base station.
• The field “Collections” is only mandatory for standalone deployed sensors.
• The field “ID” is a unique identifier which is used as deviceID for data uploads.
• The fields in the other pages of the form, e. g. Contacts, are not mandatory but nice to have, e. g. Data provider.

The AMMOD data portal uses the API of the AWI web service registry to retrieve all information of the AMMOD collection. The collection AMMOD has a persistent ID named “collectionID” which is included in the environmental variables of the AMMOD data portal.

An AMMOD base station is considered a non-movable piece of hardware which keeps the same geographical location till it is dismissed. In contrast, AMMOD sensors can be:

• moved to another base station. In this case they need to be re-assigned as a subdevice of the new base station in the sensor registry
• operated without a base station as free-floating devices.

10.2.1.2 Data requirements

Metadata files are a mandatory requirement for any data file upload. Each REST API POST request to the AMMOD data portal metadata endpoint must have a minimum of 2 files, one of which must be the metadata file. Only the telemetry endpoint of the API accepts just one file which contains the base station or sensor telemetry record.

To ease the integration and development process, JSON schemas for the metadata and telemetry file are publicly accessible at the following repository: https://gitlab-pe.gwdg.de/gfbio/ammod-examples-schemas. The repository provides the full schema documentation with examples of data and their metadata. Figure 4 shows the mandatory key-value pair included in the schema of the metadata file (Figure 4A) and telemetry file (Figure 4B). Data providers can add any base station or sensor specific key-value pairs to enrich the description of the data in the metadata and telemetry file. Those optional key-value pairs are not checked against any schema and their validity and correctness is the responsibility of the data providers.

Figure 4.

Example of JSON schema for metadata (A) and telemetry (B).

• An existing GFBio account (‘Login with your GFBio Account’)
• An existing GWDG account (‘Login with your GFBio Account’)
• Your institutional account, provided your institution is part of the DFN AAI (‘Login using DFN AAI’)
• Create a new GFBio account (follow the link in the SSO window)

The portal will automatically detect for which service you don’t have permissions to interact with the data (you should see a lock on the services where access is not granted). By clicking any of the locked services it pop-ups a message with the instructions to request the specific access. After a first login, the user is available in the website administration page and the admin can then set the required permissions. Please note that as the production and staging systems are two identical but independent systems, the login and permissions parameters are not synchronised among them.

The storage component of the AMMOD data portal has been sized according to the estimated volume of data being generated during the project pilot phase. Accordingly, the storage is set up to an initial amount of 5 TB. This is also backed up in two different locations making a total size of 15 TB. These limitations are given by the cloud service providers hosting the application. For the runtime of the AMMOD project, the data portal is deployed using the cloud computing services offered by the German Network for Bioinformatics Infrastructure (de.NBI) and by the storage in the GWDG. The current storage space is not a hard limit, and the cloud service providers offer the possibility to attach new volume to the running instance. Nevertheless, this is not an automated process and the used space needs to be monitored and managed accordingly.

As a long term solution, the data uploaded to the AMMOD portal will be transferred to the NFDI4Biodiversity core storage. Dedicated APIs to upload/download the data from the core storage are already prepared and tested. They can be deployed once a stable version of the NFDI4Biodiversity core storage is available.

Although the system has been proofed and tested under operational circumstances, bugs and errors can show up unexpectedly anytime. To continue to operate the AMMOD data portal in a resource-saving way, the performance monitoring and error tracking application Sentry (https://sentry.io/) is integrated in the AMMOD data portal. Errors are automatically collected and sent to the Sentry instance, and the admin gets informed via email immediately when something is wrong. Moreover, users can report bugs/errors using the “contact us” feature on the AMMOD website and an issue is automatically created in the software development environment.

The system has been designed to be ready to scale in case the number of users and base stations increases. At the moment the workload in terms of requests for data uploads is not considered a critical point. Indeed the workload in terms of requests for data download could increase exponentially in case the data is public. Therefore, users have no limits on the amount of files they can download, but only a 1 GB data volume per request. Once the archive with the download request is generated and the link to download the data has been sent to the user, the link is alive for 24 h. Afterwards, the link expires and the archive file is deleted from the server. These limitations might change in the future when the AMMOD data portal will be integrated with the NFDI Core Storage.

The AMMOD Data Portal allows data to be shared mainly between project members and possibly authorized collaborators. Only the status and telemetry data of the stations is publicly available. The data portal is neither a long-term archive, nor a data publication platform. To make any of the collected scientific data public, it is recommended to deposit it on one or more of the long-term GFBio data centers (https://www.gfbio.org/data-centers). This can be done using the GFBio Data Submission System (https://submissions.gfbio.org).