RFR: Regex exponential backtracking issue --- more cleanup/tuning
Xueming Shen
xueming.shen at oracle.com
Tue Mar 8 17:45:41 UTC 2016
Hi,
While waiting patiently for someone to help review the proposal for the
exponential
backtracking issue [1] I went ahead replacing those "CharProperty
constant nodes" with
the IntPredicate. We were hoping having closure back then when working
on those
CharProperty classes, which ended up with those make()/clone(). Now it
might be the
time to replace it with what we wanted at the beginning.
http://cr.openjdk.java.net/~sherman/regexClosure/webrev.02/
Here are the notes about the changes
(1) pulled out the "broken" printNodeTree (for debugging) from the
Pattern. This one does
not work as expected for a while . I do have a working copy and
have to put it in every
time I need debug the engine. So now I replaced the printNoteTree
with working one
and putting it at a separate class j.u.regex.PrintPattern, which
now can print out the
clean and complete node tree of the pattern. For example,
Pattern: [a-z0-9]+|ABCDEFG
0: <Start>
1: <Branch>
2: <CharPropertyGreedy +>
3: <Union>
4: <Range[a-z]>
5: <Range[0-9]>
<-branch.separator->
6: <Slice "ABCDEFG">
7: </Branch>
8: <END>
(2) the optimization for the greedy repetition of a "CharProperty",
which parse the
greedy repetition on a single "CharProperty", such as
\p{IsGreek}+, or the most
commonly used .* into a single/smooth loop node.
from
Pattern: \p{IsGreek}+
0: <Start>
1: <Curly GREEDY + >
2: <Script GREEK>
</Curly>
3: <END>
to
Pattern: \p{IsGreek}+
0: <Start>
1: <CharPropertyGreedy Script GREEK+>
2: <END>
The simple jmh benchmark [2] indicates it is about 50%+, especially
for those no-match
case.
(3) the optimization for the "union" of various individual "char" inside
a chracter class
[...], usch as. [ABCDEF]. For a regex like [a-zABCDEF], now the
engine generates
the nodes like
Pattern: [a-zABCDEF]
0: <Start>
1: <Union>
2: <Union>
3: <Union>
4: <Union>
5: <Union>
6: <Union>
7: <Range[a-z]>
8: <Bits [ A B C D E F]>
8: <Bits [ A B C D E F]>
8: <Bits [ A B C D E F]>
8: <Bits [ A B C D E F]>
8: <Bits [ A B C D E F]>
8: <Bits [ A B C D E F]>
9: <END>
with the optimization it generate (which it should)
Pattern: [a-zABCDEF]
0: <Start>
1: <Union>
2: <Range[a-z]>
3: <Bits [ A B C D E F]>
4: <END>
The jmh benchmark [2] also indicates it is much faster, especially
for those no-match
case.
(4) replaced those "constant" CharProperty nodes with IntPredicate (the
main change :-)).
if have time I might go further up to replace the
""CharProperty.isSatisfiedBy" with a function
"predicate" directly ... in "class" case, we actually don't really
need a "node", the only thing
we really care about is the "predicate" and their combination,
only one node for each "class".
oh, there is another one
(5) I just threw in the change for the "j.u.regex: Negated Character
Classes" [3]
(I can take it out, if anyone has trouble with this)
Thanks,
Sherman
[1]
http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-March/039269.html
[2] http://cr.openjdk.java.net/~sherman/regexClosure/MyBenchmark.java
[3]
http://mail.openjdk.java.net/pipermail/core-libs-dev/2011-June/006957.html
More information about the core-libs-dev
mailing list