Help with regular expression

For QSyntaxHighlighter I need regular expression which still cannot create. Rules are:

- word delimiters are:

Code:

---

" ' ; . , : ( ) { } [ ]

---

or <any whyte space>
- all sequences containing any other character, placed between delimeters are words, they must match to pattern only if they are completely equal to pattern
- subwords inside words (i.e. "sub" inside "submarine" or "prosubmitter" or "subsub") must not match to pattern

The rule must be single, not a set of rules. This is mandatory because of word pattern is taken from unknown list of words. It is only known that words cannot contain delimeters but all other characters they can contain. Proper words are:

Code:

---

if 
=
!=
==
+
-
hello_world
cdt*12ad
+++---
12/5/1

---

and so on. Again - they must match as whole words.

I tried several variants. Most complex part is with word match. If I create pattern like this:

Code:

---

"\\b" + pattern + "\\b"

---

then only equal words match but words like "++" do not match. If I create pattern like:

Code:

---

"(" + pattern + ")"

---

then any "+" or similar words match but even subwords match like this: "stop" becomes highlighted inside word "ifstopped".

Anybody familiar with regular expressions, please help me. The "virtual beer" will be granted.

(.*)(?= ( delim1 | delim2 | ... cont) )

then check for exact match
run indexin and for though the captured texts i would think the correct word would show up in index 1.

hope this helps

For instant regex tests i use this web tool, (make sure to check "explain" this will break down and tell you what you regex is saying )
http://myregextester.com/index.php

I don't have time to look this up, so you'll want to check syntax, but something like

^[" ' ; . , : ( ) { }\ [\ ]](if|==|!=|=|+|-|hello_world|<whatever else>)^[" ' ; . , : ( ) { }\ [\ ]]

should get you started. Note that the order of the items in the or'd middle clause is important; you generally want the more complex expressions which contain following simpler expressions to come first so they get matched before their simpler counterpart. The enclosing parentheses will let you extract the bit that matched. Note that members of the OR clause can themselves be REs; you need to pay close attention to the matching rules in such cases, though.

It would probably be simpler to tokenize on the (rather huge) list of delimiters, then simply check string equality against your list of keywords rather than using REs in this case.

The sequence of patterns cannot be used. It must work in the cycle on patterns list like this:

Code:

---

foreach( QString pattern, wordsQStringList )
{
    rule.pattern = QRegExp( "\\b(" + pattern + ")\\b" );
    rule.format = format;
    highlightRules.append( rule );
}

---

This is mandatory because of other highlighting algorithm parts. The very complex rule will slow down redrawing of large texts, up to 5000-6000 lines.

That is why I'm still confused

Then do the same thing suggested above, but in a loop.

I find it very hard to believe, though, that RE evaluation will be slower than a loop checking against all possibilities, even if the loop short-circuits. Especially the way you've formulated it here; you'll wind up re-compiling the RE on each iteration.

Quote:

---

Then do the same thing suggested above, but in a loop.

---

But HOW? This

Code:

---

foreach( QString pattern, wordsQStringList )
{
    rule.pattern = QRegExp( "^[\"';.,:(){}\[\]](" + pattern + ")^[\"';.,:(){}\[\]]" );
    rule.format = format;
    highlightRules.append( rule );
}

---

doesn't work.

About the speed - it would be better of course test which would be faster. This greatly depends from internal indexIn() implementation. But this idea use it cyclically taken from native Qt examples.

Number of words could be logically unlimited (now there about 200).

1. speed is O(n) you shouldnt be complaining.
2. each punctuational delimiter should be pretended with a '\'
3.have you even given thought about my implementation?

from what i gather you a looking for a "word" surrounded by some sort of delimiters?
and only capture Exact Matches?
word = sub, thus capture sub but not submarine?

Code:

---

QEegExp exp = QString( " exp " );
foreach( word, wordlist)
{
  if( exp.exactMatch(word) )
  {
     do something with the matched "word"
  }
}

---

Read up on QRegExp, reserved characters ect:
http://doc.qt.nokia.com/latest/qregexp.html

What is expected input just a giant string of characters that you need to split up?
what is expected output?

Quote:

---

What is expected input just a giant string of characters that you need to split up?

---

a program code on specific (not widely known) programming language placed inside QPlainTextEdit

Quote:

---

what is expected output?

---

highlighted keywords of this language, they are placed inside wordsQStringList - this is mandatory and cannot be changed (the most important reason is - number of keywords can be different and they can be written on different human languages, not only by Latin characters and can include not only letters or digits, it is imported from outside highlighter and must be used inside highlighter)

highlighting is implemented as

Code:

---

void Syntaxer::highlightBlock(const QString &text)
{
    foreach( HighRule rule, highlightRules )
    {
        QRegExp expression( rule.pattern );
        int index = expression.indexIn( text );
        while( index >= 0 )
        {  // probably there is bug in indexIn, it returns zero even if text not found
            int length = expression.matchedLength();
            if( !length ) 
                break;
            setFormat( index, length, rule.format );
            index = expression.indexIn( text, index + length );
        }
    }
}

---

as recommended in Qt examples. The exactMatch cannot be used cause text can contain different number of keywords.

all other things work fine, they are debugged and cannot be changed - I only need create a rule to highlight those keywords, but I'm not a keen in Regular Expressions

I would just recommend sitting down and reading the QT entry on QRegExp

And Play around with some on this simple tester: http://myregextester.com/index.php

I read this all carefully. If I could make expression by myself I would not ask for advice. I'm already tired after hours of attempts. I have to LOTS of other things but I stalled on this.

If you're trying to parse a language, this is probably the worst way imaginable to do it. Go read about lex and yacc if you want to do this properly.

Quote:

---

The exactMatch cannot be used cause text can contain different number of keywords.

---

So what? Every keyword is separated by delimiters and is considered alone. ExactMatch is EXACTLY what you want to use. Any of the solutions already discussed above will work just fine, although the resulting code will be a maintenance nightmare. To reiterate:

1) Split your string into individual keywords.

2) Match each keyword against the list of target keywords.

Show your code for step one. This should be a one-liner in Qt, or a very small loop in C/C++. Provide an actual sample of text and target list; they way these keep changing in your descriptions makes me almost certain there's more to know here.

Quote:

---

1) Split your string into individual keywords.

2) Match each keyword against the list of target keywords.

---

Not applicable, this will too slow. The solution must dance not from keywords but from delimiters. They are known and have very limited number. All other character sequences are keywords.

But exactly the code to find delimiters can be almost the same as for step 1). And it can be based on indexIn() call.

You have no idea at all how "slow" this approach will be, because right now the performance of YOUR algorithm is infinitely bad - it produces absolutely no results no matter how long you wait.

How about actually IMPLEMENTING something instead of whining about speed you can't even properly measure because you have nothing to compare it to? Only then will you be able to tell if performance is acceptable or not, and only then will you be able to compare performance of one approach over another.

Hint: you are making this simple problem much, much, much too hard. Stop generating excuses and generate some code, instead.

I'm not seeing any actual code, text or keyword examples. Obviously, the code doesn't exist, but the text and keyword examples are critical to understanding what your problem actually is, given that your descriptions are vague and continue to change.

I described problem and described a possible solution which I need. If you cannot help - then just do not troll me... Ignoring.

Quote:

---

Originally Posted by Gourmet

Not applicable, this will too slow. The solution must dance not from keywords but from delimiters. They are known and have very limited number. All other character sequences are keywords.

---

QStringList keywords = inputTextLine.split( QRegExp( all delimiters or'd) )

Ta freaking Da.
No one is trolling you here, Six is correct you keep complaining, and making excuses. If you are not willing to learn via trial and error you will not learn at all. thus making me wonder why you are taking a programming class

I'm not taking programming class. I'm experienced enough in C/C++ and OOP after hundreeds of thousands working lines since 1989. Currently I debug up to 400-500 lines of C++ code per day using wide set of Qt classes. But I never used Perl-like regular expressions. That is why I asked for help but not to try "teach" me something... If I told - I need this kind of solution - that means only: I need this kind of solution. Not other kind but THIS KIND. A pattern to find keyword in string with known delimiters. I did not ask for help in coding. All other suggestions I could engineer by myself and not ask for help. If the solution I asked for is impossible - just tell: "it is impossible". Then I'll find other solution by myself without help. Or tell: "Sorry I cant' help you".

Do not drop a wood bar if you have lifebuoy - drop bar only if you don't have lifebuoy. And notify about it mandatory. Otherwise drop lifebuoy but not bar. And never shout: "Teach swim yourself".

I'm going to deal with this lack of appreciation, Six and I have given you multiple way to do this, If you are not going to even try then I'm done.

I learned Regex on the fly, took me less than one hour to pick up on the simplicity of regexs.

Step one: split the code line (write one regex for this, my previous post)
Step two: see if a word is a keyword( write another regex for this )
Step three move on to next word or line. ( loops!)

Code:

---

Execution Time(sec.):
0.000009
 
Raw Match Pattern:
(.*)(\"|\'|\;|\.|\,|\:|\(|\)|\{|\}|\[|\])(.*)?
 
Match Pattern Explanation:
The regular expression:
 
(?-imsx:(.*)("|'|;|\.|,|:|\(|\)|\{|\}|\[|\])(.*)?)
 
matches as follows:
 
NODE                     EXPLANATION
----------------------------------------------------------------------
(?-imsx:                 group, but do not capture (case-sensitive)
                         (with ^ and $ matching normally) (with . not
                         matching \n) (matching whitespace and #
                         normally):
----------------------------------------------------------------------
  (                        group and capture to \1:
----------------------------------------------------------------------
    .*                       any character except \n (0 or more times
                             (matching the most amount possible))
----------------------------------------------------------------------
  )                        end of \1
----------------------------------------------------------------------
  (                        group and capture to \2:
----------------------------------------------------------------------
    "                        '"'
----------------------------------------------------------------------
   |                        OR
----------------------------------------------------------------------
    '                        '\''
----------------------------------------------------------------------
   |                        OR
----------------------------------------------------------------------
    ;                        ';'
----------------------------------------------------------------------
   |                        OR
----------------------------------------------------------------------
    \.                       '.'
----------------------------------------------------------------------
   |                        OR
----------------------------------------------------------------------
    ,                        ','
----------------------------------------------------------------------
   |                        OR
----------------------------------------------------------------------
    :                        ':'
----------------------------------------------------------------------
   |                        OR
----------------------------------------------------------------------
    \(                       '('
----------------------------------------------------------------------
   |                        OR
----------------------------------------------------------------------
    \)                       ')'
----------------------------------------------------------------------
   |                        OR
----------------------------------------------------------------------
    \{                       '{'
----------------------------------------------------------------------
   |                        OR
----------------------------------------------------------------------
    \}                       '}'
----------------------------------------------------------------------
   |                        OR
----------------------------------------------------------------------
    \[                       '['
----------------------------------------------------------------------
   |                        OR
----------------------------------------------------------------------
    \]                       ']'
----------------------------------------------------------------------
  )                        end of \2
----------------------------------------------------------------------
  (                        group and capture to \3 (optional
                           (matching the most amount possible)):
----------------------------------------------------------------------
    .*                       any character except \n (0 or more times
                             (matching the most amount possible))
----------------------------------------------------------------------
  )?                       end of \3 (NOTE: because you're using a
                           quantifier on this capture, only the LAST
                           repetition of the captured pattern will be
                           stored in \3)
----------------------------------------------------------------------
)                        end of grouping
----------------------------------------------------------------------
 
$matches Array:
(
    [0] => Array
        (
            [0] => hello:goodbye
        )
 
    [1] => Array
        (
            [0] => hello
        )
 
    [2] => Array
        (
            [0] => :
        )
 
    [3] => Array
        (
            [0] => goodbye
 
**ignore  "?-imsx:" that is looking at the possible flags you can set

---

Looks like you all did not read problem carefully... Your suggestions are not applicable. The main task is highlight keywords in the text. That means - not the presence of word in text must be confirmed. Instead the position of text to highlight must be encountered. Split list is absolutely useless in this case. It only can confirm or decline the presence of keyword inside text. But cannot give it's index.

Again - I have a list of keywords. They must be found in QString to know their positions inside this string. The set of delimiters is also known. Anybody more familiar can help?

BTW: looks like both previous "helpers" even don't know how highlighter in Qt works...

I like how you just throw out what we have inputted. good luck.

Your input does not give me anything useful except possibly this: (.*)(\"|\'|\;|\.|\,|\:|\(|\)|\{|\}|\[|\])(.*)?
But this is subject to check.