Discussion on LoG - Based operators for real-time text detection
No.19_Dec 2020|Số 19 – Tháng 12 năm 2020|p.47-56
TẠP CHÍ KHOA HỌC ĐẠI HỌC TÂN TRÀO
ISSN: 2354 - 1431
DISCUSSION ON LOG - BASED OPERATORS
FOR REAL-TIME TEXT DETECTION
Dinh Cong Nguyen1,*, PhD
1 Faculty of Information Technologies and Communication, Hong Duc University.
No 565 Quang Trung Street - Dong Ve Ward - Thanh Hoa City.
Abstract:
Article info
In this paper methods for real-time text detection in camera-based images are
presented, having a particular focus on the Laplacian of Gaussian (LoG)
operators. These methods are discussed with a specific focus on the aspects of
computational complexity and robustness. Some illustrative results and
baseline experiments are given to characterize the methods. Moreover, we
provide comments on the improvements of the methods to the text detection
problem.
Recieved:
20/9/2020
Accepted:
10/12/2020
Keywords:
Text
detection,
LoG
operator, stroke model,
almost-Gaussian.
1. Introduction
The problem of text processing in natural
images is a core topic in the fields of image
processing (IP) and pattern recognition (PR).
Recent state-of-the-art methods and international
contests can be found in [1] and [2], respectively. A
key problem is to make the methods being time-
efficient in order to embed into devices to support
real-time processing [3] [4] [5].
elongation, orientation and stroke width variation,
etc. as illustrated in Figure 1. This makes difficult
the detection problem. Therefore, various
approaches have been investigated in the literature
to design real-time and robust methods.
The recent works on the topic drive the text
processing as a blob detection problem with the
maximally stable extremal regions (MSER) [3], [5]
and the LoG-based operators [6], [8], [10], [4],
[12]. MSER looks for the local intensity extrema
The real-time systems in the [1] [3], [4] [6], [7],
[8], [9], [10] apply the strategy of two stages
composing of detection and recognition. The
detection localizes the text components at a low
complexity level and groups them into text
candidate regions before classification. The
objective is to get a perfect recall for the detection
with a maximum precision for optimization of the
recognition. The two-stage strategy differs from the
end-to-end strategy, that applies template/feature
matching with classification using high-level
models for text entities [11]. The text elements in
natural images present specific shapes with
and applies
a
watershed-like segmentation
algorithm for detection. The algorithm is processed
in a linear time complexity. It copes well with
background/foreground regions but is sensitive to
blurring. The Laplacian of Gaussian (LoG) operator
is a blob detector, but can be tuned to a stroke
detector with scale and orientation for better
characterization of text elements [10], [4].
Recently, LoG estimators have been proposed at a
linear-time complexity [13], [14] making the
operator competitive with MSER.
Dinh Cong Nguyen/ No.19_Dec 2020|p.47-56
Figure 1. Example of text elements/characters in images [12]
Figure 2. A characterization of different methods in the paper.
This paper gives several key contributions.
Optimization is obtained with the difference of
Gaussian (DoG) and difference-of-offset-Gaussian
(DooG) reformulation of the operators, then
estimation with almost-Gaussian components.
operators for blob detection. The adaptation of the
LoG operator to stroke/text detection will be
meaning of symbols used in the paper.
We focus only on text detection phase, we
bring together all the recent trends of the LoG-
based operators dealing with adaptation to the text
detection problem.
We discuss and concentrate on how to
optimize these operators with real-time constraints.
Figure 2 characterizes different methods in the
paper with key sections.
The baseline LoG operator is reformulated
into the stroke model paradigm and generalized
LoG (gLoG) for scale and adaptive rotation.
2. Baseline LoG Operators
One of the standard approaches for differential blob detector is found by LoG based on the Gaussian
function. The multivariate Gaussian function, with a vectorial notation, is given in Eq. (1).
Dinh Cong Nguyen/ No.19_Dec 2020|p.47-56
(
)
( )
( |
)
( )
( ) | |
√
In the two-dimensional case, n = 2, p is a point
deviations in x, y. Considering = , μ is null
and a scalar notation, the Gaussian function Eq. (1)
becomes Eq. (2).
and μ is a centroid. Σ is the diagonal covariance
matrix with
the inverse and |Σ| the
determinant, where ,
are the standard
(
| )
( )
(
| )
The LoG is a compound operator resulting of the Laplacian of Eq. (3).
(
| )
(
| )
(
| )
(
)
(3)
The LoG-filtered image h(x, y) Eq. (4) is obtained by the global convolution between the initial image f(x, y)
(
| )
and the LoG operator
(
)
(
| )
(
)
(
| )
(
)
( )
( )
LoG function can be approximated by means of DoG as Eq. (5) with relation among ( ) as Eq. (6).
(
| )
(
| )
(
| )
( )
(
)
( )
where can be presented as with k a parameter, resulting in the DoG formulation Eq. (7).
(
)
(
| ) )
(
|
(
)
( )
scale σ increases [15]. As illustrated in Figure 4,
this motivates application of the LoG operator for
text [10] [4].
As the scale of LoG is relatively low, we tend
to use LoG in order to detect edges with zero-
crossing. In contrast, blob-like structures will be
converged at some scales to local extrema when the
Figure 4. Blob-based detection for text detection with a LoG operator with σ = 2.3.
[10]) and LoG kernel reformulation [4].
3. The LoG Operators for Text Detection
The LoG operator has been applied in different
works for text detection in [10] [4] [12] [14]. In this
paper, we will explore recent trends on this topic
dealing with adaptation of the operator to the text
detection problem. This includes of the control of
standard deviation parameters σ (stroke model [6]
3.1. The Stroke Model
A crucial problem with the LoG operator for
blob detection is the control of the scale parameter
σ [12]. When the object to detect is a text element/
character, the LoG operator can be driven as a
stroke detector where the parameter σ is able to be
Dinh Cong Nguyen/ No.19_Dec 2020|p.47-56
derived from the stroke width parameter w. This is
minimal/maximal derivatives of the convolution
product. Assuming that these minimum/maximums
are located at the center of the stroke w/2, we can
present the standard deviation σ as a function σ =
f(w). These aspects will be developed here.
presented as the stroke model in literature.
Figure 5 illustrates the model. The general idea
is to look for the convolution response between a
LoG-based operator and a stroke signal model as
unit step function. We can express then the
Figure 5. LoG responses at different scales to (a) a step function (b) a boxcar function [14].
parameter, the convolution product with the LoG
Assuming the image signal as a function
Π(x) (considering 1-D case as discussed in [10])
operator ( ) is given in Eq. (8).
Π(x) the step function Eq. (9) and a as a constant
( )
(
)( )
(
)
( )
( )
∫
(
)
{
( )
( )
(
)
As is located at , the
From derivative of Eq. (10), the local
extremal optimum is obtained as Eq. (11) with k a
parameter.
(
)
( )
( )
( )
convolution product ⨂ over
x equals the summation at centered at .
( | )
Approximately
√
( )
( | ) as DoG function, the result of Eq. (8) is
reformulated into Eq. (10).
Discussion:
( )
( | )
∫ (
As given in Eq. (11) and shown in Figure 5(a),
it is seen that locations are dependent on the σ
(
)
( | ))
parameter. With x = x + w/2 the middle of the
2
0
stroke and goes to Eq. (11), we can get the
optimum scale and operator response Eq. (12)
√
√
√
( (
) (
)) ( )
where erf(x) is the Gauss error function erf(x) =
. The optimum/extremal responses
∫
Dinh Cong Nguyen/ No.19_Dec 2020|p.47-56
(these aspects are not proven in the paper [10], but
illustrated with experiments) of the DoG operator
appear at the middle of the stroke w/2 with a
setting parameter σ . However, the operator is
s
limited in detecting blobs with general elliptical
shapes and is not able to estimate the orientation of
the detected blobs. Indeed, the conventional LoG
operator is rotational symmetric, i.e., the σ is set to
be equal for both x and y coordinates. The Figure
6(a) illustrates this problem, as the character is
rotated, variations appear in the stroke width
resulting in the lowest responses of the operator
accurate scaling parameter σ . This response
s
decreases while shifting the scaling parameter σ
around σ optimum Figure 5(b).
s
3.2 The Generalized LoG Operator
The LoG (either DoG) operator has good
performances in locating the middle of 2-D near
circular blobs, with a proper standard deviation
Figure 6. (a) LoG responses at scale 휎푠= f(w) with a regular and a rotated character (b) gLoG response at
scale 휎푥 = f(푤 ), 휎푦 = f((푤 ) with a rotated character.
To address this problem the LoG operator is
generalized to detect elliptical and rotated shapes
Figure 6(b). This makes the operator robust to the
detection cases with rotation and shifts the operator
for detection of Haar-like features. For
simplification, we refer the generalized operator as
gLoG as suggested in [15]. At best of our
knowledge, only the paper [16] has investigated
this issue for text detection. Recent contributions on
the gLoG detector for natural images are found in
[15].
Let us g(x, y| σ , σ , θ) as 2-D oriented
x
y
Gaussian function with form as Eq. (13),
( )
( | )
( )
with a, b trigonometric functions to control the
resulting from Eq. (13). The convolution products
of gLoG with the given image will be used to
determine the shape and the orientation of blobs.
shape and the orientation with standard deviations
and
orientation
θ.
The
gLoG
( | ) is obtained by Eq. (14)
( | ) ( | ) ( | )
( )
Discussion
Figure 7. Approximations of (a) 푔푥 with 퐷표표퐺푥 (b) 푔푥푥with 퐷표표퐺푥푥 reformulations.
Dinh Cong Nguyen/ No.19_Dec 2020|p.47-56
For optimization, difference-of-offset-Gaussian
between Gaussian functions with relatively small
offset distances in Figure 7. The first derivative in x
dimension of the 2-D oriented Gaussian function
Eq. (13) is given in Eq. (15), where a, b, c
parameters are defined in Eq. (13). The DooG
function Eq. (16) can approximate the Gaussian
derivative function Eq. (15).
(DooG) operator is considered, which was first
introduced by Young [17]. Basically, DooG
function is designed by using Eq. (13) with offset
values , as the distance between two
Gaussian kernels [18]. It could be explained that the
derivatives of Gaussian function are
mathematically closely equal to discrete difference
a
Figure 8. (a) a character, responses in color map of (b) the LoG operator (c) the BSV
operator (d) the BSV after hysteresis thresholding.
(
)
(
)
( | ) ( | )
( | ) ( | ) ( | )
( | )
( )
The DooG operator can be extended to the second derivative from the x or y dimensions Eq. (17). These
operators approximate the second order derivatives of Gaussian
.
( | ) ( | ) ( | )
( | ) (17)
With ( | ) and ( | ) formulations, we can approximate the
gLoG operator Eq. (14) as given in Eq. (18).
( | ) ( | ) ( | )
( )
3.3 The BSV Operator
location and a null response in the in-between edge
area Figure 8(b), the BSV operator still guaranties a
no null response Figure 8(c). Then, similar to edge
detector the stroke elements can be obtained with
hysteresis thresholding Figure 8(d).
The BSV operator [4] is a LoG look-like
operator for stroke detection. It differs from the
blob-based strategy with LoG, that targets optimum
response (10) with the scale parameter
The BSV operator is close to Laplacian
formulation Eq. (3). It results in the total
differential d of an image function f(x, y) convolved
with a δ(x, y) operator Eq. (19).
Eq. (12). The operator processes as an edge detector
with a zero-crossing operation, where the optimum
scale for edge detection ≪ . Whereas the
LoG operator produces a strong response at an edge
(
)
(
)
(
)
(
)
( )
( )
Using the linearity property, the compound
operator BSV(x, y) = d(δ(x, y)) can be achieved in
formulation of Biot-Savart law into an image
convolution operator as described from original
paper [4] in detail.
(
)
(
)
Eq. (20) with as defined in
Eq. (21). This operator is expressed from the the
(
)
(
)
(
)
(
)
( )
( )
( )
(
)
( )
( )
( )
Dinh Cong Nguyen/ No.19_Dec 2020|p.47-56
produce a LoG look-like function as Eq. (23) with
Discussion
(
| )
( | )
the Gaussian derivatives.
A convolution with the BSV operator is close to
a derivative product, but with specific steps and
averaging. When a Gaussian averaging product is
embedded Eq. (22), the BSV operator tends to
Compared to the LoG, the BSV operator enhances
the central part of the kernel that maintains a
response in the in-between edge area
(
)
( (
| )
(
) )
(
)
(
)
(
)
|
( )
( )
(
)
(
| )
(
| )
| ( ) ( )
The compound operator BSV(x, y) of Eq. (20) is
such as shifting the
complexity to O(Nω).
not separable. The real-time property is coming
from the operator size, as we have ≪ .
If the DoG operator introduces
a
main
However, optimization could be obtained with the
non-compound form of the operator (these aspects
are not discussed in [4]). The Gaussian derivatives
optimization compared to the LoG operator,
however the complexity O(Nω) is not parameter-
free. The recent trends with camera devices (e.g.
smartphones, tablets) are to process up to 10-Mpx
for image streaming at 30 to 60 frames per second
DoG operator can guarantee the frame rate at a low
resolution only (less then 2-Mpx). If a low
resolution is sufficient for simple text scene image
Figure 9(a), it introduces character degradations
with complex scene images Figure 9(b).
(
| )
(
| )
can be approximated
with DooG operators Eq. (16) then almost-Gaussian
are functions close to Haar-like features that could
be approximated with boxcar operators [13].
4 Discussion on Real-time LoG Operators
The baseline approach to process a LoG
operator is the convolution product. The LoG
function (3) is discretized to get a mask g of size ω
For optimization, the DoG operator can be
estimated with almost-Gaussian functions [13] [20].
This enters in an estimator cascade methodology
× ω, applied in the product . The size
of the mask is dependent on the σ parameter
̂
̂
LoG ≈ DoG ≈ , where is the DoG
estimator. Specifically, repeated filtering with the
averaging filters can be used to approximate a
Gaussian filter, as given below Eq. (24) and shown
[19].
(the typical size is for a full coverage of the
function [19]), requiring a complexity O(N )
with N the image size (in pixels). Optimization is
obtained with the DoG function Eq. (5) that can be
implemented with separable filters of size 1 × ω
Figure 9. (a) image with text from with processing time /FPS of DoG/almost- Gaussian operators at different resolutions
with parameters 휎푠 (11) (b) degradations of text/characters at low resolutions with a complex scene image.
Dinh Cong Nguyen/ No.19_Dec 2020|p.47-56
(
| )
(
)
̂ ∑
( )
I
n
Figure 10. Approximation process (a) approximation of Gaussian function after the successive
averaging (b) DoG can be obtained from approximation of Gaussian.
( )
(
)
the Eq. (24) is a given box filter
( )
function having a predefined size. The quality of
approximation is based on the number of repeated
filtering n, certainly no more than 6. It can be
justified by Eq. (25) in order to obtain
approximation of a Gaussian, as presented in [19],
where ω is the width of the averaging filter.
From approximation of Gaussian in Eq. (24), it
becomes possible to approximate the DoG operator
̂
by in (26) with two sets of box filter function.
Figure 10(b) gives a plot of Eq. (26).
̂
(
| ) )
(
)
(
| )
(
̂ ̂
(
)
(
)
(
)
(
)
∑
∑
( )
operator is controlled through the stroke model
paradigm for scale-invariance. The gLoG operator
[15] guaranties the rotation and contrast-invariance.
All these operators are symmetric except the gLoG
operator. The symmetric operators detect the
medical axes of characters that produces an
important number of keypoint candidates. These
keypoints must be post-processed for grouping. The
gLoG operator relaxes this constraint, it the
processes with a full primitive detection. Therefore,
it is a time-consuming operator and is minimally
compatible with a real-time strategy. However, it
could be approximated by the DooG operator, even
(
)
(
)
Obviously, the products
from Eq. (26) is able to be obtained with integral
image at complexity O(N). As a result, approximation
of DoG is possibly achieved with 2n accesses of
integral image, it therefore is parameter free.
The DoG filter is then approximated as a linear
combination of several box filters . Then, box
coefficients must be found to minimize the
approximation error. In [13], this is presented as an
L1 regularized least-square problem that can be
solved with an optimization algorithm (e.g. LASSO
as detailed on the optimization aspects). The
experiments in [13] report that DoG estimator
̂
with the operator. This point has been little
achieves an acceleration at low scales
explored in the literature, it then could be a
promising solution.
[1.5, 3.1], while maintaining a low average mean
square error compared to the DoG. Figure 9(a)
gives the processing time of the estimator over the
5 Conclusions and Perspectives
This paper has presented how the LoG operators
can be set and adapted for text detection problem
and made real-time with an estimator cascade
methodology. Some main perspectives and
challenges remain. Firstly, the LoG operators for
text detection have mainly been investigated with
symmetric model. However, little work exists on
the generalization case (i.e. gLoG operator). The
different image resolutions and scales .
The BSV operator [4] is the edge-based
operator while applying a hybrid strategy that
generates a blob detection from an edge detection
using a LoG look-like function. Although they get a
sake of time-efficiency, the edge-based operators
perform a poor detection as an average. The LoG
Dinh Cong Nguyen/ No.19_Dec 2020|p.47-56
generalization can turn the operator into a stroke
[11] J. Matas and L. Neumann, "Real-time lexicon-
free scene text localization and recognition,"
PAMI, vol. 38.9, pp. 1872-1885, 2016.
detection for a better detection accuracy. Next, the
real-time methodology with estimator cascade
offers intermediate acceleration factors (≃ ×2 to
×4). It processes as a Full-Search (FS) method in
the spatial domain with the fast estimation of the
operator product. Similar to template matching,
further acceleration could be obtained with FS-
equivalent methods.
[12] D. Nguyen, M. Delalandre, D. Conte and T.
Pham, "Perfor- mance evaluation of real-time
and scale-invariant LoG operators for text
detection.," VISAPP, pp. 344-353, 2019.
[13] V. Fragoso, G. Srivastava, A. Nagar, Z. Li, K.
Park and M. Turk, "Cascade of Box (CABOX)
Bibliography
Filters
for
Optimal
Scale
Space
[1] Q. Ye and D. Doermann, "A survey Text
detection and recognition in imagery," PAMI,
vol. 37.7, pp. 1480-1500, 2015.
Approximation," CVPR, pp. 126-131.
[14] D. Nguyen, M. Delalandre, D. Conte and T.
Pham, "Fast RT‐LoG operator for scene text
detection," JRTIP, 2020.
[2] R. Gomez and B. Shi, "ICDAR2017 robust
reading challenge on COCO-Text," ICDAR,
pp. 1435-1443, 2017.
[15] H. Kong, H. Akakin and S. Sarma, "A
generalized Laplacian of Gaussian filter for
blob detection and its applications," Cyber,
vol. 43.6, pp. 1719-1733, 2013.
[3] H. Yang and C. Wang, "An Improved System
For Real-Time Scene Text Recognition," Proc.
Mul., pp. 657-660, 2015.
[16] N. Makhfi and O. Bannay, "Scale-space
approach for character segmentation in
scanned images of arabic document. J. . : 444
(2016)," Theo. App. Infor. Tech, vol. 94.2,
2016.
[4] X. Girones and C. Julia, "Real-Time Text
Localization in Natural Scene Images Using a
Linear Spatial Filter," ICDAR, pp. 1261-1268,
2017.
[5] S. Deshpande and R. Shriram, "Real time text
detection and recognition on hand held objects
to assist blind people," Proc. Dyn. Opt. Tech,
pp. 1020-1024, 2016.
[17] R. Young, "Gaussian derivative theory of
spatial vision: analysis of cortical cell
receptive field line-weighting profiles,"
Motors Research Laboratories, 1985.
[6] B. Epshtein, E. Ofek and Y. Wexler,
"Detecting text in natural scenes with stroke
width transform," CVPR, pp. 2963-2970, 2010.
[18] W. Ma and M. B.S., "EdgeFlow: a technique
for
boundary
detection
and
image
segmentation," TIP, vol. 9.8, pp. 1375-1388,
2000.
[7] L. Neumann and J. Matas, "Real-time scene
text localization and regconition," CVPR, pp.
3538-3545, 2012.
[19] P. Kovesi, "Fast almost-gaussian filtering,"
Dig. Ima. Comp. Tech, pp. 21-125, 2010.
[8] L. Neumann and J. Matas, "Scene text
localization and regconition with oriented
stroke detection," ICCV, pp. 97-104, 2013.
[20] M. Grabner, H. Grabner and H. Bischof, "Fast
approximated SIFT," ACCV, pp. 918-927,
2006.
[9] L. Gomez and D. Karatzas, "MSER-based
real-time text detection and tracking," in ICPR,
2014.
[21] D. Sen and S. Pal, "Gradient histogram:
Thresholding in a region of interest for edge
detection," IVC, vol. 28.4, pp. 677-695, 2010.
[10] Y. Liu, D. Zhang, Y. Zhang and S. Lin, "Real-
time scene text detection based on stroke
model," ICPR, pp. 3116-3120, 2014.
Dinh Cong Nguyen/ No.19_Dec 2020|p.47-56
THẢO LUẬN VỀ CÁC TOÁN TỬ DỰA TRÊN LoG
ĐỂ PHÁT HIỆN VĂN BẢN THEO THỜI GIAN THỰC
Dinh Cong Nguyen PhD
Thông tin bài viết
Ngày nhận bài:
Tóm tắt
Trong bài báo này trình bày các phương pháp phát hiện văn bản thời gian thực
trong hình ảnh dựa trên máy ảnh, tập trung đặc biệt vào toán tử Laplacian of
Gaussian (LoG). Các phương pháp này được thảo luận với sự tập trung cụ thể
vào các khía cạnh của tính phức tạp và tính mạnh mẽ. Một số kết quả minh họa
và các thí nghiệm cơ bản được đưa ra để mô tả đặc điểm của các phương pháp.
Hơn nữa, bài báo cũng cung cấp nhận xét về những cải tiến của các phương
pháp đối với vấn đề phát hiện văn bản.
20/9/2020
Ngày duyệt đăng:
10/12/2020
Từ khóa:
Phát hiện văn bản, toán tử
LoG, mô hình đột quỵ,
almost-Gaussian.
10 trang
21400
Free
Bạn đang xem tài liệu "Discussion on LoG - Based operators for real-time text detection", để tải tài liệu gốc về máy hãy click vào nút Download ở trên
File đính kèm:
- discussion_on_log_based_operators_for_real_time_text_detecti.pdf
