Audio beehive monitoring based on IoT-AI techniques: A survey and perspective
ISSN 2588-1299
VJAS 2020; 3(1): 530-540
Vietnam Journal
of Agricultural
Sciences
Audio Beehive Monitoring Based on IoT-AI
Techniques: A Survey and Perspective
Nguyen Huu Du1, Nguyen Doan Dong2, Vu Thi Luu3, Nguyen
Van Hoang3, Pham Hong Thai4, Phan Thanh Ngoc4, Nguyen
Viet Long5 & Phan Thi Thu Hong3
1Department of Mathematics, Faculty of Information Technology, Vietnam National
University of Agriculture, Hanoi 131000, Vietnam
2Department of Software Engineering, Faculty of Information Technology, Vietnam
National University of Agriculture, Hanoi 131000, Vietnam
3Department of Computer Science, Faculty of Information Technology, Vietnam National
University of Agriculture, Hanoi 131000, Vietnam
4Research Center for Tropical Bees and Beekeeping, Vietnam National University of
Agriculture, Hanoi 131000, Vietnam
5Center for Agricultural Innovation, Vietnam National University of Agriculture, Hanoi
131000, Vietnam
Abstract
In beekeeping, monitoring beehives plays an important role in
making sure bee colonies stay healthy and in reducing negative
effects that could happen in the colonies. A large number of studies
have been carried out to improve the performance of monitoring
beehives from the traditional manual methods. Most importantly, the
application of artificial intelligence (AI) technologies in recent times
have led to significant effects in the monitoring process. These new
methods, however, have not yet been investigated or applied in
Vietnam. To understand the use of AI-based technologies in the
automatic monitoring of beehives, this paper provides a survey on
beehive monitoring systems based on audio data and AI techniques.
Opportunities and perspectives for the applications of these
techniques in audio-based monitoring beehive in Vietnam are also
discussed.
Keywords
artificial intelligence, IoT, Beehive monitoring, Beehive sound
recognition, bioacoustics
Introduction
Received: May 14, 2020
Accepted: June 5, 2020
The honey bee (Apis mellifera) has a crucial value to the
agricultural industry around the world. Along with commercial
productions such as honey, beeswax, and royal jelly for human
consumption, bees play an essential part in pollinating plants. It is
estimated that roughly one-third of all crops grown for human
consumption are pollinated by honey bees (Klein et al., 2007). In
recent years, many factors have led to a decline of honey bee
Correspondence to
ORCID
Phan Thi Thu Hong
3721
530
Vietnam Journal of Agricultural Sciences
Nguyen Huu Du et al. (2020)
colonies, and this fact requires continuous and
extensive monitoring to investigate the factors
that may negatively affect the life cycle of bees.
failing queens, and airborne toxicants
(Bromenshenk et al., 2009). In practice, many
beekeepers listen to their hives to find out the
state of their honey bee colonies. They can
recognize the changes in buzzing sounds
produced by stressed colonies. However, they
may not always be able to determine the exact
causes of the changes without hive inspections.
The new technology-based methods are very
helpful and effective in these situations. They can
provide the beekeepers information about the
state of the hives so that they can take
precautionary measures to restore the hives’
health. By these methods, IoT-based beehive
monitoring systems are designed to capture large
amounts of useful information about the
behaviors of the bees. They consist of several
physical devices embedded in a hive with
sensors, actuators, electronics, software, and a
network connection that enable recording of the
buzzing sounds made by the honey bees,
changing them into electrical signals, and then
passing them to a programmable logic controller.
Since the data collected at this stage is raw,
extraction algorithms are needed to extract
critical information from these raw data before
applying artificial intelligence (AI) algorithms
for each specific problem related to monitoring
beehives. It is well documented in the literature
that analyzing bee buzzing data is an effective
way to monitor the health of beehives. A large
number of studies have been carried out on this
topic. Thorough descriptions of bee sounds and
their characteristics were given in papers by
Bencsik et al. (2007) and Zacepins et al. (2016).
Dietlein (1985) analyzed the acoustic signals
produced by bees and gave definitions of
frequency bands. A simple method based on
acoustic technology to monitor beehives is
presented in an article by Vancata (1995). By
using computational analysis of the acoustic
vibrations captured with accelerometers, Ramsey
et al. (2017) presented a method to identify a
honey bee swarming event. The problem of
investigating audio signals of honey bee swarms
was also considered in the published work of
Schultz et al. (2008). A distributed audio
monitoring system for apiaries was developed by
Mezquida & Martinez (2009). Recently, AI
Monitoring beehives to gather information
about the honey bee's status, activities, and
behavior is one of the most important tasks in the
process of beekeeping. It is not simply a check of
the beehive for pollen, nectar, and brood
quantities, but also an inspection of the presence
of pests or diseases such as viruses, bacteria,
fungi, and mite infections. Regular checks are
critical for finding and preventing these
problems. In practice, this task is usually
performed by manual methods. However,
traditional methods lead to many factors that
make inspections not very effective. For
example, the manual method is time-consuming,
and it depends strongly on the experience and
knowledge of beekeepers. Several symptoms
may not be recognized by amateur or
inexperienced beekeepers. It is also impossible
for people to continuously monitor beehives to
promptly detect problems related to a honey bee
colony’s status by this manual method. An
external dismantling for internal inspection
creates stress and panic in bees, and can disrupt
the life cycles of bee colonies. Frequent
disturbances can be a stressful factor and
introduce anxiety to the swarm. In many
production areas, the hives may be located far
away from human habitats, resulting in
additional transportation costs. Thus, modern
technology-based methods to reduce these
disadvantages of the traditional method in
monitoring bee colonies are encouraged. The
general idea of these new approaches is to use a
system of electronic devices attached to each
beehive to collect data. The collected data are
then analyzed and processed to obtain important
information that allows beekeepers to monitor
beehives remotely without interrupting the life
cycles of the honey bee colonies.
Among several types of data collected from
bee colonies, the buzzing sound plays a vital role
in automated beehive monitoring. This is
because bee buzzing carries information on
colony behavior and phenology. For example,
honey bees emit specific sounds when they are
exposed to stressors such as predatory mites,
531
Audio beehive monitoring based on IoT-AI techniques: a survey and perspective
algorithms have emerged as effective method in
recognizing and classifying beehive audio
samples. For example, Kulyukin et al. (2018)
provided a study on deep learning and machine
learning methods for audio beehive monitoring.
The obtained results from this study confirmed
that AI algorithms could be used in real
electronic beehive monitors. As in many other
fields, these algorithms could bring a revolution
in automatic beehive monitoring, and they have
become a hot trend in recent times. To the best of
our knowledge, the research and development of
audio-based beehive monitoring systems have
not been considered in Vietnam. As a result,
these new technologies have not been applied in
any Vietnamese bee farms. This paper aims to
provide a survey of audio-based beehive
monitoring techniques that could be useful for
researchers or beekeepers who want to apply
advanced technologies to their work. While there
other types of data collection methods to
consider, like image data, the systems to collect
this type of data requires more. In addition, a
camera for capturing bee images is much more
expensive than the sound sensors, and the humid
air and high water vapor conditions in Vietnam
may lead to unclear images unless these devices
are designed on demand. Therefore, we restricted
this paper to the studies related to audio-based
monitoring systems and the corresponding AI
algorithms. The opportunities and perspectives
of the applications of these technologies are also
considered and discussed.
were considered in the first step: monitoring
beehive, honey bee monitoring, audio beehive
monitoring, audio-based recognition, and bee
sound analysis, among others.
In addition, we focused mainly on journal
papers, however, other sources like proceedings,
books, and PhD/MSc dissertations, which
appeared in these papers, were also taken into
account.
Results
IoT-based hardware
The first step of any data collection project
is to design and install a system to collect the data
from beehives. This problem has been addressed
by many researchers. Gil-Lebrero et al. (2017)
provided an overview of systems designed for
monitoring beehives from the early years of the
twentieth century to recent years. It can be seen
from this overview that the monitoring systems
have greatly progressed thanks to the
development and applications of the IoT
technologies. An IoT-based monitoring system is
made up of embedded devices that can acquire
data from different sensors, process them, and
communicate with a computer with a cloud
database. According to the discussion in the
paper by Zganks (2019), two main architectures
of an IoT-based monitoring service can be
designed in which the crucial point is the IoT
module and its capacity for processing power and
memory. The first one corresponds to the IoT
module with low processing power and memory,
i.e., its function is only to capture the bee signals
in the beehive and transmit them over a wireless
connection to a backend server in a cloud (or on
the edge), where the AI algorithms take part. This
architecture has a variant where the IoT module
can carry out feature extraction and transmit
feature vectors instead of audio signals. The
second one corresponds to the case when the IoT
module has enough processing power and
memory to run AI algorithms locally in the
beehive. The second architecture requires a more
complex design and unlimited wireless
connectivity, and thus, has a higher cost.
Materials and Methods
Among the three methods used to investigate
the state of knowledge in a particular field,
namely, “Delphi”, “meta-analysis”, and “content
analysis” (Li & Cavusgil, 1995), we utilized the
content analysis method in this study which is
able to categorize and analyze references in a
systematic way. By applying this method, we
implemented two general steps. The first step
was to determine the survey sources and the
procedures for searching the relevant documents.
The second step was to identify the categories for
classifying and analyzing the references from the
first step. Moreover, the following keywords
A general structure of the beehive-
monitoring system, that meets several significant
Vietnam Journal of Agricultural Sciences
532
Nguyen Huu Du et al. (2020)
requirements, including (1) using a minimally
invasive method, (2) being operational in remote
areas for long periods, and (3) allowing real-time
monitoring was described in detail by Gil
Lebrero et al. (2017). The authors also presented
descriptions of the hardware and software of the
wireless node. Although the system was
designed for capturing temperature and humidity
data, it can be considered as a general structure
for systems capturing other data like sounds by
replacing the temperature and humidity sensors
with sound sensors in the adapter board. Several
specific systems have been designed to collect
bee sound data in the literature based on this
general structure. For example, the Raspberry Pi
Model B (2016), which is a family of single-
board computers, approximately the size of a
credit card developed by the non-profit
Raspberry Pi Foundation, was considered in a
range of published papers (Schurischuster et al.,
2016; Zacepins et al., 2016; Antonio, 2017;
Crawford, 2017). A multi-sensor electronic
beehive monitoring system, called BeePi, has
also been quite popular (Kulyukin et al., 2016;
Amlathe, 2018; Kulyukin, 2019). Each BeePi
monitor consists of exclusively off-the-shelf
components: a Raspberry Pi computer, a
miniature camera, a microphone splitter, four
microphones connected to a splitter, a solar
panel, a temperature sensor, a battery, and a
hardware clock (Kulyukin et al., 2018). Another
hardware platform for bee sound data acquisition
was used by Cecchi et al. (2019) and Qandour et
al. (2014).
Feature extraction
Feature extraction plays an essential role in
machine learning pipelines. After being extracted
from the raw data, the important features will be
fed to analytical algorithms to detect specific
problems related to the beehives. This step is
performed by using the artificial intelligence (AI)
algorithms, which are described below.
There are several approaches used to extract
meaningful features for ML models in terms of
bee sound analysis. The Mel-frequency cepstral
coefficients (MFCCs) (Davis & Mermelstein,
1980) method appears most frequently in the
literature to extract features from audio samples,
see (Amlathe, 2018; Kulyukin et al., 2018;
Zganks, 2018; Nolasco et al., 2019; Zganks,
2019). The pipeline of MFCCs implementation is
depicted in Figure 1.
Figure 1. The implementation of MFCCs
533
Audio beehive monitoring based on IoT-AI techniques: a survey and perspective
In the article by Nolasco et al. (2019),
MFCCs, Mel spectrograms, and HHTs (Huang &
Schen, 2005) are considered to represent features
for ML algorithms to recognize the presence of
the queen. There have been experiments
conducted to test various scenarios by combining
those extractors as the inputs for traditional
models and convolutional neural networks. The
best result for traditional models was the use of a
combination of MFCCs and HHTs. Zganks,
(2018) proposed an approach to recognize bee
swarm activities by applying MFCCs as an
extractor. The results showed that the approach
could be applied in a real-life setting. In another
paper by Zganks (2019), the normalization of the
cepstral mean was applied for the features
extracted by using MFCCs. Also, linear
predictive coding (LPC) was used to compare to
MFCCs. The obtained results showed that the
performance of models based on the MFCCs was
better. The study by Amlathe (2018) utilized the
combination of MFFCs, Chroma sfft, Mel
spectrograms, and Tonnetz with 193 features in
total as inputs for ML models to classify bee
buzzing, cricket chirping, and ambient noise.
This work has been extended in another study by
Kulyukin et al. (2018) in which 34 features were
extracted (i.e., temporal features of raw data,
MFCCs, and magnitudes of DFT) and reduced to
13 features by applying principal component
analysis (PCA). In order to improve the
performance of standard ML models, the features
selection techniques were also applied. Peak
frequency (PF) and the mean of Short time
Fourier transforms (STFT) were utilized in a ML
model to detect Varroa mites (Qandour et al.,
2014).
beehives has been proposed by Brundage (2012)
by analyzing the frequency of bee sounds near
the entrance of the beehive. Terenzi et al. (2019)
investigated the performance of Wavelet
transforms (WT) as a feature extraction
technique for honey bee sound classifications in
comparison to the SFTF, MFCCs, and HHTs.
The conclusion was that the performance of WT
based approaches was better than the others and
the WT coefficients can be used in ML
algorithms.
AI algorithms for audio-based beehive
monitoring
Recognition of the bee queen’s absence
The queen bee plays a very important role in
a colony. Her main mission in the colony is to
release pheromones and produce eggs. Checking
whether or not the queen bee is healthy and
capable of laying eggs is perhaps one of the most
critical tasks for beekeepers. The absence of the
queen bee means that there are no more eggs or
larvae, i.e. there are no younger bees that could
replace the older ones. As a consequence, the life
of the whole swarm will be exposed to danger
since a worker bee only lives up to 40 days and
their work is usually overtaken by another bee
after that (Tautz, 2008). That is to say, the lack of
a healthy queen bee is extremely unfavorable and
this absence should be detected as soon as it is
possible. The timely detection of the queen's
absence will lead to an accurate understanding of
the cause of the absence as well as an appropriate
intervention to introduce a new queen bee to the
colony. This is a major concern in several studies
in the literature. The acoustic-based methods,
based on spectrographic analysis and a Kohonen
self-organising map (SOM) artificial neural
network, have been used to investigate the
queenless state (Howard et al., 2013). The
obtained results showed that the relatively low-
frequency ranges for queenless hives can be
described as bulges in the frequency spectrum,
relative to normal activity, between 165Hz and
285Hz. However, the results from using a SOM
to classify the data from different hives have
been less successful. Robles-Guerrero et al.
(2017) used a logistic regression (LR) model for
honey bee buzz recognition. The sound from two
Besides using ML for recognition, bee
behaviors also can be estimated by applying the
techniques to analyze audio signals and audio-
processing. Ferrari et al. (2008) proposed a
method to detect swarming in advance by
analyzing spectrograms. The low-frequency
spectrums produced by honey bees increase
before a swarming event. Michelsen et al. (1986)
proposed a method to analyze the frequency of
wagging dance sounds and found a correlation
between bees’ dancing and the harmonics around
320 Hz. A method to assess the productivity of
Vietnam Journal of Agricultural Sciences
534
Nguyen Huu Du et al. (2020)
colonies of bees in the Mel frequency domain
was recorded. The first was from a healthy
colony with a queen and the second one was from
a hive with no queen and with a reduced
population. The obtained results showed that it
was possible to classify both characteristics
obtaining high classification rates using a
reduced set of features. However, the authors
also claimed that data analysis from more hives
are needed in order to confirm these results. A
support vector machine (SVM) with a Gaussian
kernel was utilized by Cejrowski et al. (2018) to
classify data from different states of the beehive.
The SVM was also applied by Nolasco (2018)
and Nolasci et al. (2019) but with a radial basis
function kernel. In these studies, the SVM led to
better accuracy in generalizing the unseen hives
compared to the convolutional neural networks
(CNN). However, the CNN achieved good
results in a hive-dependent scenario which
indicated the feasibility of the application of deep
learning methods to this problem.
SVM algorithm, which was used to detect the
queenless status, can also be applied to detect
swarming in honey bee colonies as stated by
Cejrowski et al. (2018). A system with Gaussian
mixture models (GMM) and HMM with different
topologies has also been proposed to analyze the
swarm classification performance (Zganks,
2019). The evaluation of the proposed methods
in monitoring bee swarm activity from the
captured input audio signal shows promising
results. However, there is a difference between
the GMM and HMM approaches in the case of a
medium topology that needs to be analyzed
further to obtain better-classifying performance.
The identification of the honey bee swarming
process by analyzing the time course of hive
vibrations based on a principal component
analysis (PCA) method was introduced in the
work of Bencsik et al. (2007).
Recognition of beehive sounds
The first fundamental task of any audio-
based beehive monitoring technology is to
recognize bee sounds and discriminate them
from non-bee sounds that might be captured. The
non-bee sounds are usually related to the
environment and events occurring in the
surroundings of the hive such as urban sounds,
rain, or other animals like crickets. Once the
systems fail to distinguish the bee sounds from
these non-bee sounds, all the work related to data
analysis for specific problems after that will also
fail. Therefore, automatically detecting sounds
produced by bees and distinguishing them from
external non-related sounds in the audio data
captured inside beehives is the main goal in a
number of studies. Amlathe (2018) used four
machine learning algorithms to train the
classification models to detect bee sounds from
audio samples, namely the LR algorithm, the K-
nearest neighbor (KNN) algorithm, the random
forest (RF) algorithm, and the SVM algorithm.
According to this study, the best accuracy for the
three-way classification problem (aiming to
classify audio samples into one of the three
categories: bee buzzing, cricket chirping, or
ambient noise, which were manually labeled)
was achieved using the random forest algorithm,
while the best accuracy for the day classification
problem (aiming to classify audio samples into
Recognition of the bee queen’s absence
Swarming, a natural process in the life of a
honey bee colony, is one of the activities in a
beehive that requires an immediate response
from beekeepers. This occurs when a large group
of honey bees leaves an established colony and
flies off to start a new colony and settle in a new
hive. Since honey bee swarms may contain
several hundred to several thousand worker bees,
a few drones, and an old queen, the results of
swarming influence the current status of the
beehive and future productivity of the colony.
Several studies have been carried out to predict
swarming moments by individuating the main
factors that could lead a hive to swarm (Seeley &
Tautz, 2001; Visscher & Seeley, 2007; Rittschof
& Seeley, 2008; Kridi et al., 2014). An overview
of the factors that are connected with swarming
and the related studies can be seen in the paper of
Ferrari et al. (2008) in which a method for
predicting the swarming period was proposed
based on labeling the bees’ sounds. Zganks
(2018) used a hidden Markov model (HMM) for
his experiment to separate the bee swarm activity
from the routine daily activity based on the open
audio data provided by the Open Source
Beehives Project (“OSBeehives”, 2020). The
535
Audio beehive monitoring based on IoT-AI techniques: a survey and perspective
one of six-time categories in a day) was achieved
using the KNN classifier. Other methods also
achieved high accuracy. This study was extended
to allow the authors to compare the performance
of these four machine learning methods with
several CNNs, i.e. the RawConvNet (the CNN
that classifies raw audio samples) and three
deeper SpectConvNets (the CNNs that classify
audio spectrogram images from audio datasets)
with different combinations of fully connected
and convolution layers (Kulyukin et al., 2018).
methods to detect Varroa mites as soon as
possible. However, most of these studies were
based on image data (Ramrez et al., 2012;
Schurischuster et al., 2016). In the work of
Qandour et al. (2014), the authors used the PCA
algorithm to narrow the features extracted from
the feature extraction step. This reduced feature
set was then put through a linear discriminant
analysis (LDA) algorithm and the SVM
algorithm. The obtained results showed that the
LDA led to the highest accuracy, while the SVM
resulted in a slightly improved computational
time. In general, both these methods were
capable of distinguishing between infected and
healthy beehive sounds.
Their experiments
indicated that
the
RawConvNet could perform better than the
SpectConvNets, while these SpectConvNets
outperformed the four machine learning model.
That is to say, adding more layers may not
necessarily result in improved classification
performance in the audio datasets considered in
this study. The authors also discussed the main
trade-offs between the ConvNets and the
machine learning methods, which was between
feature engineering and training time.
Comparing the performance of the SVM and the
CNN for beehive sound recognition was also
made based on the data recordings from the Open
Source Beehive project (Nolasco & Benetos,
2018). These experiments showed that the SVM
lead to slightly better results and less overfitting
occurred for the SVM which may indicate better
generalization capabilities for CNN. This result
is in accordance with the findings in another
paper by Nolasco et al. (2019).
Discussion
Opportunities for the application of AI
techniques in audio-based beehive monitoring
in Vietnam
In Vietnam, beekeeping provides
a
significant source of income, contributing to
improving the lives of producers, as well as
making an important contribution to Vietnam’s
export turnover. Each year, Vietnam produces
over 55,000 tons of honey and more than 1,000
tons of beeswax, of which about 85-90% of the
output is exported. Vietnam is ranked 6th in the
world and 2nd in Asia in honey exports. However,
it should also be pointed out that honey
beekeeping in Vietnam still has several
limitations, especially problems related to
modern technologies. The production process is
based purely on manual methods, including
manual inspections, leading to low productivity
and quality. In this section, we discuss some
opportunities and further perspectives of the
applications of AI techniques to improve the
quality of the process of raising and producing
honey in Vietnam.
Recognition of beehive sounds
Another application of AI techniques in
monitoring beehives based on audio data is to
detect pest infestations. This research has only
emerged recently from a study in which the
authors focused on developing a system that
could recognize the different acoustic
characteristics produced by a healthy colony and
a colony infested with Varroa mites (Qandour et
al., 2014). Within the last few decades, the global
spread of the parasitic mite Varroa destructor,
better known as the Varroa mite, has resulted in
the loss of millions of honey bee (Apis mellifera)
colonies, and it is undoubtedly the biggest mortal
threat to honey bees now (Martin, 2001). Many
studies have been carried out to find efficient
Applying the available methods to monitor
honey beehives
As discussed in the previous section, AI
algorithms have been successfully applied to
solve several important problems related to non-
invasive monitoring beehives based on the
analysis of sound generated within the beehives.
Vietnam Journal of Agricultural Sciences
536
Nguyen Huu Du et al. (2020)
These problems involve the recognition of bee
sounds, the recognition of the queen’s absence,
and the recognition of swarming. In several
studies, not only the methods but also the designs
of the systems have been described in detail
(Cejrowski et al., 2018; Kulyukin et al., 2018).
Moreover, the devices and sensors recording the
bee sounds are reasonably priced and suitable for
the conditions of production in Vietnam.
Therefore, these results can be applied to develop
AI techniques-based systems for classifying
beehive states from bee sounds in bee farms in
Vietnam. This will provide beekeepers with a
continuous and autonomous analysis of their
beehives. The architecture of our proposed
approach is illustrated in Figure 2.
term memory (LSTM) networks (Hochreiter &
Schmidhuber, 1997), and LSTM autoencoders
(Marchi et al., 2017), among others. These are
promising algorithms for analyzing bee sounds
since they have been shown to have very high
performances in speech recognition. A problem
that needs to be considered in using these
algorithms is how to build a bee sound dataset
large enough to train the classifying model.
Automatic identification of other important
beehive states
The recognition of Varroa mites is just one
of several problems related to the recognition of
pest infestations based on audio data. After the
study of Qandour et al. (2014), we could not find
any other studies related to this problem in the
literature. That is to say, more research is needed
in this area. One could use other AI algorithms
(as discussed in the previous subsection) or one
could combine sound data analyses with other
types of data like humidity and temperature
within a beehive to improve the performance of
the monitoring process (Ferrari et al., 2008). The
system to collect these data would need to have a
simple design and low cost, consistent with the
production conditions in Vietnam.
Exploring new AI algorithms for bee sound
analysis
We have found from reviewing the
references related to methods for monitoring
beehives based on acoustic analyses in the
literature that most of these studies focused on
the problem of feature extraction from raw data.
Although several AI algorithms, which are listed
in Table 1, have been proposed to process the
features from the extraction step, these
algorithms were just some standard machine
learning algorithms and some popular deep
learning algorithms. Many other AI algorithms
are well documented to have very good
performance in recognizing and classifying
audio data such as the Boltzmann machine
(Salakhutdinov & Hinton, 2009), deep recurrent
neural networks (Phan et al., 2017), long short
Conclusions
An audio-based beehive monitoring system
includes many components, ranging from
designing an IoT-based system to the use of
feature extraction and AI algorithms. Among
them, the AI algorithms contribute significantly to
the performance and accuracy of the monitoring
Figure 1. The architecture of audio-based beehive monitoring system
537
Audio beehive monitoring based on IoT-AI techniques: a survey and perspective
Table 1. A list of AI algorithms with applications for audio-based beehive monitoring
AI
Application Scenarios
Feature Extraction
MFCCs
References
algorithms
Robles-Guerrero et al.
MFFCs, Chroma sfft, Mel spectrograms, and
Tonnetz
(2017)
Recognition of the bee queen’s
LR
absence
Amlathe (2018)
MFFCs, temporal features, and DFT
magnitudes
Kulyukin et al. (2018)
MFFCs, Chroma sfft, Mel spectrograms, and
Tonnetz
Amlathe (2018)
KNN
RF
Recognition of bee sounds
MFFCs, temporal features, and DFT
magnitudes
Kulyukin et al. (2018)
MFFCs, Chroma sfft, Mel spectrograms, and
Tonnetz
Amlathe (2018)
Recognition of bee sounds
Recognition of swarming
MFFCs, temporal features, and DFT
magnitudes
Kulyukin et al. (2018)
MFCCs
Zganks (2018)
Zganks (2019)
HMM
PCA
LDA
MFCCs and LPC
Recognition of swarming
Short Frequency Spectra
PF and STFT
Bencsik (2007)
Recognition of pest infestations
Qandour et al. (2014)
Recognition of pest infestations
PF and STFT
Qandour et al. (2014)
Nolasco et al. (20
MFCCs, Mel spectrograms, and HHT
LPC
Nolasco (2018)
Recognition of the bee queen’s
Nolasco & Benetos
(2018)
absence
MFFCs, Chroma sfft, Mel spectrograms, and
Tonnetz
Recognition of the bee queen’s
Cejrowski et al. (2018)
Amlathe (2018)
absence
SVM
MFFCs, temporal features, and DFT
magnitudes
Kulyukin et al. (2018)
Recognition of swarming
MFCC and Mel spectrograms
Nolasco & Benetos
(2018)
Recognition of pest infestations
LPC
PF and STFT
Cejrowski et al. (2018)
Qandour et al. (2014)
MFCCs, Mel spectrograms, and HHT
Nolasco et al. (2019)
Kulyukin et al. (2018)
Recognition of the bee queen’s
MFFCs, temporal features, and DFT
magnitudes
absence
CNN
SOM
Nolasco & Benetos
(2018)
Recognition of bee sounds
MFCCs, Mel spectrograms, and HHT
Recognition of the bee queen’s
StockWell Transform
Howard (2013)
absence
system. This paper has presented a survey on the
AI techniques used in an audio-based beehive
monitoring system. Several opportunities for the
applications of these advanced techniques to
monitoring beehives in Vietnam have been
discussed. This is a promising topic worthy of
being considered to contribute to improving the
quality of monitoring honey bee colonies in
Vietnam.
The authors thank the anonymous referees
for their insightful and valuable suggestions
which helped to improve the quality of the final
manuscript.
References
Amlathe P. (2018). Standard machine learning techniques
in audio beehive monitoring: Classification of audio
samples with logistic regression, K-nearest neighbor,
random forest and support vector machine. Master of
Science, Utah State University, 57 pages.
Acknowledgements
Vietnam Journal of Agricultural Sciences
538
Nguyen Huu Du et al. (2020)
Antonio R. G., Tonatiuh S. A., Efr´en, G. R. & Carlos E. G.
(2017). Frequency Analysis of Honey Bee Buzz for
Hochreiter S. & Schmidhuber J. (1997). Long short-term
memory. Neural Computation. 9(8): 1735-1780.
Automatic Recognition of Health Status:
Preliminary Study. Research in Computing Science.
142: 89-98.
A
Howard D., Duran O., Hunter G. & Stebel K. (2013). Signal
Processing the acoustics of honeybees (APIS
MELLIFERA) to identify the” queenless” state in
Hives. Proceedings of the Institute of Acoustics. 35(1):
290-297..
Bencsik M., Bencsik J., Baxter M., Lucian A., Romieu J. &
Millet M. (2011). Identification of the honey bee
swarming process by analysing the time course of hive
vibrations. Computers and Electronics in Agriculture.
76(1): 44-50.
Huang N. E. & Schen S. (2005). An Introduction to Hilbert-
Huang Transform and Its Applications. In: Huang N.
E. & Schen S. (Eds.). Interdisciplinary Mathematical
Sciences, volume 5: Hilbert-Huang Transform and Its
Applications. ISBN: 978-981-256-376-7.
Bromenshenk J. J., Henderson C. B., Seccomb R. A., Rice
S. D. & Etter R. T. (2009). Honey bee acoustic
recording and analysis system for monitoring hive
health. U.S. Patent. No: 7,549,907 B2.
Klein A.-M., Vaissiere B. E., Cane J. H., Steffan-Dewenter
I., Cunningham S. A., Kremen C. & Tscharntke T.
(2007). Importance of pollinators in changing
landscapes for world crops. Proceedings of the Royal
Society B: Biological Sciences. 274(1608): 303-313.
Brundage T. J. (2012). Acoustic sensor for beehive
monitoring. U.S. Patent 8: 152-590.
Cecchi S., Terenzi A., Orcioni S., Spinsante S., Primiani V.
M., Moglie F., Ruschioni S., Mattei C., Riolo P. &
Isidoro N. (2019). Multi-sensor platform for real time
measurements of honey bee hive parameters. IOP
Conference Series: Earth and Environmental Science.
275: 012016. DOI:10.1088/1755-1315/275/1/012016.
Kulyukin V., Mukherjee S. & Amlathe P. (2018). Toward
audio beehive monitoring: Deep learning vs. standard
machine learning in classifying beehive audio samples.
Applied Sciences. 8(9): 1573.
Kulyukin V. A., Putnam M. & Reka S. K. (2016).
Digitizing buzzing signals into A440 piano note
sequences and estimating forager traffic levels from
images in solar-powered, electronic beehive
monitoring. Proceedings of the International
MultiConference of Engineers and Computer
Scientists, March 16-18, 2016, Hong Kong.
Cejrowski T., Szymański J., Mora H. & Gil D. (2018).
Detection of the bee queen presence using sound
analysis. Asian Conference on Intelligent Information
and Database Systems. Springer. 297-306.
Chollet F. (2017). Deep Learning with Python. Manning
Publications. 384 pages.
Kulyukin V. A. (2019). BeePi: A Multisensor Electronic
Beehive Monitor [Online]. Retrieved from
meetai/beepi-a-multisensor-electronic-beehive-
monitor on May 14, 2020.
Crawford M. (2017). Automated collection of honey bee
hive data using the Raspberry Pi, Master of science,
Appalachian State University, 69 pages.
Davis S. & Mermelstein P. Comparison of parametric
representations for monosyllabic word recognition in
continuously spoken sentences. IEEE Transactions on
Acoustics, Speech, and Signal Processing. 28: 357-
366.
Li T. & Cavusgil S. T. (1995). A classification and
assessment of research streams in international
marketing. International Business Review. 4(3): 251-
277.
Dietlein D. G. (1985). A method for remote monitoring of
activity of honeybee colonies by sound analysis.
Journal of Apicultural Research. 24(3): 176-183.
Marchi E., Vesperini F., Squartini S. & Schuller B. (2017).
Deep recurrent neural network-based autoencoders for
acoustic novelty detection. Computational Intelligence
and Neuroscience. DOI: 10.1155/2017/4694860.
Kridi D. S., Carvalho C. G. N. d. & Gomes D. G. (2014). A
predictive algorithm for mitigate swarming bees
through proactive monitoring via wireless sensor
networks. Proceedings of the 11th ACM symposium
on Performance evaluation of wireless ad hoc, sensor,
& ubiquitous networks. 41-47.
Martin S. J. (2001). The role of Varroa and viral pathogens
in the collapse of honeybee colonies: a modelling
approach. Journal of Applied Ecology. 38(5): 1082-
1093.
Ferrari S., Silva M., Guarino M. & Berckmans D. (2008).
Monitoring of swarming sounds in bee hives for early
detection of the swarming period. Computers and
electronics in agriculture. 64(1): 72-77.
Mezquida D. A. & Martínez J. L. (2009). Platform for bee-
hives monitoring based on sound analysis. A perpetual
warehouse for swarm s daily activity. Spanish journal
of agricultural research. 7(4): 824-828.
Gil-Lebrero S., Quiles-Latorre F. J., Ortiz-López M.,
Sánchez-Ruiz V., Gámiz-López V. & Luna-Rodríguez
J. J. (2017). Honey bee colonies remote monitoring
system. Sensors. 17(1): 55-76.
Michelsen A., Kirchner W. H. & Lindauer M. (1986).
Sound and vibrational signals in the dance language of
the honeybee, Apis mellifera. Behavioral ecology and
sociobiology. 18(3): 207-212.
Haste T., Tibshirani R. & Friedman J. (2009). The elements
of statistical learning: Data Mining, Inference, and
Prediction. Springer-Verlag New York. 745 pages.
DOI: 10.1007/978-0-387-84858-7.
Nolasco I. & Benetos E. (2018). To bee or not to bee:
Investigating machine learning approaches for beehive
sound recognition. Workshop on Detection and
Classification of Acoustic Scenes and Events
539
Audio beehive monitoring based on IoT-AI techniques: a survey and perspective
(DCASE), New York, USA, October 25-26, 2018:
133-137.
mechanism of flight guidance in honeybee swarms:
subtle guides or streaker bees? The Journal of
Experimental Biology. 211(20): 3287-3295.
Nolasco I. (2018). Audio-based beehive state recognition,
Master thesis, Queen Mary University of London.
Schurischuster S., Zambanini S., Kampel M. & Lamp B.
(2016). Sensor study for monitoring varroa mites on
honey bees (apis mellifera). Visual Observation and
Analysis of Vertebrate and Insect Behavior Workshop,
Cancun Mexico, December 4, 2016.
Nolasco I., Terenzi A., Cecchi S., Orcioni S., Bear H. L. &
Benetos E. (2019). Audio-based identification of
beehive states. IEEE International Conference on
Acoustics, Speech and Signal Processing (ICASSP),
Brighton, UK, May 12 – 17, 2019: 8256-8260.
Seeley T. D. & Tautz J. (2001). Worker piping in honey bee
swarms and its role in preparing for liftoff. Journal of
Comparative Physiology. 187: 667-676.
OSBeehives platform (2018). OSBeehives | BuzzBox Hive
Health Monitor & Beekeeping App. Retrieved from
Tautz J. (2008). The Buzz about Bees: Biology of a
Superorganism. Springer, Berlin, Heidelberg.
Phan H., Koch P., Katzberg F., Maass M., Mazur R. &
Mertins A. (2017). Audio scene classification with
deep recurrent neural networks. arXiv preprint
arXiv:1703.04770.
Terenzi A., Cecchi S., Orcioni S. & Piazza F. (2019).
Features Extraction Applied to the Analysis of the
Sounds Emitted by Honey Bees in a Beehive. 11th
International Symposium on Image and Signal
Processing and Analysis (ISPA). 3-8.
Qandour A., Ahmad I., Habibi D. & Leppard M. (2014).
Remote beehive monitoring using acoustic signals.
Acoustics Australia. 42(3): 204-209.
Vancata I. (1995). Using acoustic technology to monitor
your hives. American Bee Journal. 135(9): 615-618.
Ramírez M., Prendas J. P., Travieso C. M., Calderón R. &
Salas O. (2012). Detection of the mite Varroa
destructor in honey bee cells by video sequence
processing. 2012 IEEE 16th International Conference
on Intelligent Engineering Systems (INES). IEEE.
103-108.
Visscher P. K. & Seeley T. D. (2007). Coordinating a group
departure: who produces the piping signals on
honeybee swarms? Behavioral Ecology and
Sociobiology. 61(10): 1615-1621.
Zacepins A., Kviesis A., Stalidzans E., Liepniece M. &
Jurijs Meitalovs (2016). Remote detection of the
swarming of honey bee colonies by single-point
temperature monitoring. Biosystems Engineering.
148: 76-80.
Ramsey M., Bencsik M. & Newton M. I. (2017). Long-term
trends in the honeybee ‘whooping signal’ revealed by
automated detection. PLoS One. 12(7): e0181736.
Raspberry Pi Foundation (2016). Raspberry Pi Products".
Retrieved from https://www.raspberrypi.org/products/
on May 14, 2020.
Zacepins A., Kviesis A., Ahrendt P., Richter U., Tekin S.
& Durgun M. (2016). Beekeeping in the future - Smart
apiary managemen. 17th International Carpathian
Control Conference (ICCC). 808-812.
Rittschof C. C. & Seeley T. D. (2008). The buzz-run: how
honeybees signal ‘Time to go!’. Animal Behaviour.
75(1): 189-197.
Zganks A. (2018). Acoustic monitoring and classification
of bee swarm activity using MFCC feature extraction
and HMM acoustic modeling. ELEKTRO. 1-4.
Salakhutdinov R. & Hinton G. (2009). Deep boltzmann
machines. Artificial intelligence and statistics. 448-
455.
Zganks A. (2019). Bee Swarm Activity Acoustic
Classification for an IoT-Based Farm Service. Sensors.
20(1).: 21-35. DOI: 10.3390/s20010021.
Schultz K. M., Passino K. M. & Seeley T. D. (2008). The
Vietnam Journal of Agricultural Sciences
540
11 trang
18560
Free
Bạn đang xem tài liệu "Audio beehive monitoring based on IoT-AI techniques: A survey and perspective", để tải tài liệu gốc về máy hãy click vào nút Download ở trên
File đính kèm:
- audio_beehive_monitoring_based_on_iot_ai_techniques_a_survey.pdf
