QCodeEdit

[fullmetalcoder]

hi, I've had planned quite a while ago to rewrite the text editing module of Edyuk and, as I was bored with writing Edyuk docs last week, I started thinking about that.

QCodeEdit is meant to be a framework providing :

• a powerful and flexible widget mixing a QTextEdit and various panels
• an easily extendable framework for "language definitions" : i.e. each programming language is associated with one or more file extensions and gets its own parenthesis matcher, text indenter and syntax highlighter
• an wide set of powerful generic components that interfaces between XML definitions and "language framework"

The archive I'm releasing is just a first draft and currently features only the core of the language framework and the generic highlighting engine.

For simplicity's sake it's gonna be possible to attach the language framework to a QTextEdit or to QCodeEdit indifferently.

The beta is bundled with a minimal example and an XML definition for C++ (with Doxygen stuff)

Have fun and feed me back.

[elcuco]

Anyone tred this code...?

Anyway, I tested your code. In order to make it run I needed to apply this patch:

diff -ru qcodeedit-0.1.0-beta-orig/example/example.pro qcodeedit-0.1.0-beta/example/example.pro
--- qcodeedit-0.1.0-beta-orig/example/example.pro       2006-07-28 16:37:06.000000000 +0300
+++ qcodeedit-0.1.0-beta/example/example.pro    2006-08-01 23:03:25.000000000 +0300
@@ -4,8 +4,10 @@
 
 TEMPLATE = app
 TARGET = example
-
+INCLUDEPATH +=  ../include ../src ../src/editor ../src/language ../src/generic
+DEFINES += _QCODE_EDIT_GENERIC_
 CONFIG += release qcodeedit
+LIBS += -L../src/ -lqcodeedit

diff -ru qcodeedit-0.1.0-beta-orig/example/example.pro qcodeedit-0.1.0-beta/example/example.pro
--- qcodeedit-0.1.0-beta-orig/example/example.pro       2006-07-28 16:37:06.000000000 +0300
+++ qcodeedit-0.1.0-beta/example/example.pro    2006-08-01 23:03:25.000000000 +0300
@@ -4,8 +4,10 @@

 TEMPLATE = app
 TARGET = example
-
+INCLUDEPATH +=  ../include ../src ../src/editor ../src/language ../src/generic
+DEFINES += _QCODE_EDIT_GENERIC_
 CONFIG += release qcodeedit
+LIBS += -L../src/ -lqcodeedit

To copy to clipboard, switch view to plain text mode 

I also needed to copy the *.so file into the example dir. When you type text the application does work, but when pasting large portions of text (qwidget.cpp was my test case) or load it via the command line the text does not get painted.

About the code itself:
The idea of having different contexts in the language definitions if cool. This means that different color can be applied to the same word depending of it's location on the text.

I am not sure that putting the colors definitions inside the syntax declaration is a good idea: you are not able to modify the colors of your syntax. For example, people which have a pink desktop will have problems with your code, since the text editor will always be with white background.

I am not sure that re-inventing the wheel and defining a new syntax declaration is a good idea. If you choose another project language definitions are will gain a lot of syntaxes for free, and you now only have to code an engine to display them. For example:

Sharp develop also use XML for describing the syntaxes:
http://www.icsharpcode.net/OpenSource/SD/

GtkSourceView also uses XML (see also my minimal implementation for this definitions)
http://gtksourceview.sourceforge.net/

And also notepad++ (win32 application):
http://notepad-plus.sourceforge.net/uk/site.htm

Another suggestion can be using plain "ini" files instead XML. This will make the construction on new syntax definition files much easier.

[fullmetalcoder]

Anyone tred this code...?

Anyway, I tested your code. In order to make it run I needed to apply this patch:

diff -ru qcodeedit-0.1.0-beta-orig/example/example.pro qcodeedit-0.1.0-beta/example/example.pro
--- qcodeedit-0.1.0-beta-orig/example/example.pro       2006-07-28 16:37:06.000000000 +0300
+++ qcodeedit-0.1.0-beta/example/example.pro    2006-08-01 23:03:25.000000000 +0300
@@ -4,8 +4,10 @@
 
 TEMPLATE = app
 TARGET = example
-
+INCLUDEPATH +=  ../include ../src ../src/editor ../src/language ../src/generic
+DEFINES += _QCODE_EDIT_GENERIC_
 CONFIG += release qcodeedit
+LIBS += -L../src/ -lqcodeedit

diff -ru qcodeedit-0.1.0-beta-orig/example/example.pro qcodeedit-0.1.0-beta/example/example.pro
--- qcodeedit-0.1.0-beta-orig/example/example.pro       2006-07-28 16:37:06.000000000 +0300
+++ qcodeedit-0.1.0-beta/example/example.pro    2006-08-01 23:03:25.000000000 +0300
@@ -4,8 +4,10 @@

 TEMPLATE = app
 TARGET = example
-
+INCLUDEPATH +=  ../include ../src ../src/editor ../src/language ../src/generic
+DEFINES += _QCODE_EDIT_GENERIC_
 CONFIG += release qcodeedit
+LIBS += -L../src/ -lqcodeedit

To copy to clipboard, switch view to plain text mode 

I also needed to copy the *.so file into the example dir. When you type text the application does work, but when pasting large portions of text (qwidget.cpp was my test case) or load it via the command line the text does not get painted.

About the code itself:
The idea of having different contexts in the language definitions if cool. This means that different color can be applied to the same word depending of it's location on the text.

I am not sure that putting the colors definitions inside the syntax declaration is a good idea: you are not able to modify the colors of your syntax. For example, people which have a pink desktop will have problems with your code, since the text editor will always be with white background.

I am not sure that re-inventing the wheel and defining a new syntax declaration is a good idea. If you choose another project language definitions are will gain a lot of syntaxes for free, and you now only have to code an engine to display them. For example:

Sharp develop also use XML for describing the syntaxes:
http://www.icsharpcode.net/OpenSource/SD/

GtkSourceView also uses XML (see also my minimal implementation for this definitions)
http://gtksourceview.sourceforge.net/

And also notepad++ (win32 application):
http://notepad-plus.sourceforge.net/uk/site.htm

Another suggestion can be using plain "ini" files instead XML. This will make the construction on new syntax definition files much easier.

I'm pleased to have some feedback but I must inform you that the code I posted here is quite out-of-date... I've continued developping it for use with Edyuk and you can always find the latest version on the SVN trunk or (a little older one) on the latest surce package.

The syntax definition in itself still needs some work. Honestly I didn't pretend to have crafted the ultimate file format here. I just thought about one that would be human readable and has some convinience. I'm not against using other ones but as you might have noticed I'm using some rather special mechanism like the href stuff and the context aware definitions (or nested definitions). Besides if you want to support another generic definition format all you need is to reimplement the some interfaces for highlighting, paren matching, ... and also create and add proper definitions to the factory.

About the colors : I didn't had much time to think about that but you're probably right... I should place them in a separate file. I'll give it a try in a next version.

Using INI files should allow faster and a little easier syntax creation but it would also make them look ugly when using nested/context-aware highlighting definitions...

The convention in Qt is that only Trolltech issues classes with names like QClassName. Third-party developers are expected to put initials after the Q to identify it as theirs, as in QfmcCodeEdit. You can find the naming guidelines here. You are supposed to register the initials with Trolltech, but I have registered my own (ij) and have not heard from them and it has not appeared on their site.

I had never heard of that and, IMHO, it's rubbish... Long prefixes make the code longer to type and sometimes harder to understand. Moreover, everytime I use the Q prefix it's to state a self-sufficient component (apart from Qt dependencies of course ) that is meant to become someday a library well-integrated into Qt... Ultimately, if the Trolls asked me to change that I'd rather put a namespace than changing all class names...

[No-Nonsense]

I'm pleased to have some feedback but I must inform you that the code I posted here is quite out-of-date... I've continued developping it for use with Edyuk and you can always find the latest version on the SVN trunk [...]

I would like to try out the latest version of QCodeEdit as I am programming a source editor for some custom language.
I started with a QTextEdit, added a QSyntaxHighlighter and wanted to have line-numbers and later syntax completion and icons on the line-number-gutter. I just started a own editor based on QTextEdit and using various Qt editors as a base how to implement a gutter etc.

Sadly I cannot compile your source as the *.pro file is missing in ./src/lib/qcodeedit/ or do you have a separate SVN repository for QCodeEdit?

The syntax definition in itself still needs some work. Honestly I didn't pretend to have crafted the ultimate file format here. I just thought about one that would be human readable and has some convinience. I'm not against using other ones but as you might have noticed I'm using some rather special mechanism like the href stuff and the context aware definitions (or nested definitions).

You could look at the Katepart language definitions. Personally I dislike some of their syntax (some XML elements start with a lowercase char, some with uppercase...; some of the rules are overdefined, e.g. matching a decimal/hex/... can be done with a regexp instead too). But what you could add support for a context stack (or isn't this needed by your definitions?) and standard colors.

About the colors : I didn't had much time to think about that but you're probably right... I should place them in a separate file. I'll give it a try in a next version.

I would prefer the way Katepart solves this: They define some standard colors (keyword, comment, decimal, hex, float, ...). The user (programmer) can then set the colors and styles for these. In the syntax definition file you would allow to set a color or to use one of these placeholders. Then QCodeEdit would init the associated colors with some defaults the programmer can overwrite at any point with other values.
You could also extend this to letting the syntax definition define the placeholder-names and offer a way for the programmer to get a list of names and build a config dialog for the user to set the colors.

-Jens

[travlr]

-Jens

Jens, are you the same no-nonsense from Elite Trader??

[No-Nonsense]

Jens, are you the same no-nonsense from Elite Trader??

No, don't think so. What is Elite Trader?

-Jens

[travlr]

No, don't think so. What is Elite Trader?

-Jens

Oh.. Ok. I guess not.
ET is a forum where financial market traders gather.

-travlr

[fullmetalcoder]

I would like to try out the latest version of QCodeEdit as I am programming a source editor for some custom language.
I started with a QTextEdit, added a QSyntaxHighlighter and wanted to have line-numbers and later syntax completion and icons on the line-number-gutter. I just started a own editor based on QTextEdit and using various Qt editors as a base how to implement a gutter etc.

Sadly I cannot compile your source as the *.pro file is missing in ./src/lib/qcodeedit/ or do you have a separate SVN repository for QCodeEdit?

QCodeEdit is a componnent and when released as is it contains a pro file to compile it as a static or shared lib. However I thought it would be absolutely useless to keep this pro file in Edyuk repositories and packages. Instead I split the pro files into : a wrapper .pro that's quite simple and distributed only in QCodeEdit packages and a .pri file (PRoject Include) which has to be included in your project via the include() function...

If you want to start playing with QCodeEdit I really recommend to use the iclude() function at first because it will spare you the pain of compiling an lib and then linking your app to it... However if you need a sample pro file to build and install QCodeEdit as a regular lib on your system : just ask me, I'll craft you one in a few minutes...

You could look at the Katepart language definitions. Personally I dislike some of their syntax (some XML elements start with a lowercase char, some with uppercase...; some of the rules are overdefined, e.g. matching a decimal/hex/... can be done with a regexp instead too). But what you could add support for a context stack (or isn't this needed by your definitions?) and standard colors.

I don't quite understand your last sentence...

I would prefer the way Katepart solves this: They define some standard colors (keyword, comment, decimal, hex, float, ...). The user (programmer) can then set the colors and styles for these. In the syntax definition file you would allow to set a color or to use one of these placeholders. Then QCodeEdit would init the associated colors with some defaults the programmer can overwrite at any point with other values.
You could also extend this to letting the syntax definition define the placeholder-names and offer a way for the programmer to get a list of names and build a config dialog for the user to set the colors.

Yep! I got to do it that way but unfortunately I'm ATM kept rather busy by the developpment of Edyuk and such improvements will have to wait a bit.

[No-Nonsense]

QCodeEdit is a componnent and when released as is it contains a pro file to compile it as a static or shared lib.

This may be OT here, but with my simple extended QTextEdit (only line-number-gutter) I had problems to get it working as a static library. I could get it to work as a shared library (plugin for Qt Designer) or by simply adding all files to the source an header variable in the *.pro and using "promote to custom widget".

I already searched the forums here, but did not find any usefull hints. I only changed the plugin pro file and added the static keyword there. Do I have to change anything else to get this to work? Maybe I can send you my test sources?

Another problem I had was acessing member objects of my custom TextEdit (i.e. the Gutter). I always had to include the header (ok) but also add the *.h and *.cpp to the project sources & headers when using as a plugin. Is there a way to only use the library and the *.h files? Because if not the plugin makes no sense to me as I could use it directly by including all files.

If you want to start playing with QCodeEdit I really recommend to use the iclude() function at first because it will spare you the pain of compiling an lib and then linking your app to it... However if you need a sample pro file to build and install QCodeEdit as a regular lib on your system : just ask me, I'll craft you one in a few minutes...

Could you provide me a samle project please?

I don't quite understand your last sentence...

In the Kate syntax definition you define some sort of a statemachine. There is a start context and rules that can stay or change the context. Then you are able to push and pop the current context while changing it. An example is given here: http://docs.kde.org/stable/en/kdebas...ml-format.html

Thanks in advance,
-Jens

[fullmetalcoder]

Could you provide me a samle project please?

Sure! But I need an e-mail address...

In the Kate syntax definition you define some sort of a statemachine. There is a start context and rules that can stay or change the context. Then you are able to push and pop the current context while changing it. An example is given here: http://docs.kde.org/stable/en/kdebas...ml-format.html

The Kate syntax engine sounds nice and as far as I've played with it, decently fast... However the problem I have is simple : time... Unless their implementation is small enough and can easily be ported to Qt4/QCodeEdit I won't try to use their definition format because I'm quite busy ATM...

[fullmetalcoder]

Here comes the fun! I finally took a few minutes to package the last source of QcodeEdit and released them as QCodeEdit 1.0 on Edyuk's Sf.net donwload server.

Hoping this will help those who need a ready-made editing framework and motivate those asking for a new syntax engine...

[magland]

Looking forward to trying it out...

Had some trouble compiling. The readme.txt is confusing, because there are three .pro files in the main directory of distribution.

... so I tried
>> qmake qcodeedit.pro
>> make

But this sent me into a recursive situation ??
qmake -win32 -o Makefile qcodeedit.pro
qmake -win32 -o Makefile qcodeedit.pro
qmake -win32 -o Makefile qcodeedit.pro
qmake -win32 -o Makefile qcodeedit.pro
qmake -win32 -o Makefile qcodeedit.pro
qmake -win32 -o Makefile qcodeedit.pro
....

I'm using Windows with MinGW
Help?

[fullmetalcoder]

Looking forward to trying it out...
I'm using Windows with MinGW
Help?

Apologies... I made a shell script but no batch equivalent... Anyway it shouldn't be to hard... Just copy makeall.sh as makeall.bat, open it as text and remove the first line (you may also need to replace every occurence of "make" by "mingw32-make") and everything should be fine.

[magland]

Thanks, the library seemed to compile, but now I'm having trouble compiling example.pro. I get the following error. Any advice?

tmp\obj\main.o(.text+0x171): In function `Z5qMainiPPc':
C:/cstuff/qcodeedit-1.0/main.cpp:26: undefined reference to `QPlainTextEditCreat
or::id()'
tmp\obj\main.o(.text+0x192):C:/cstuff/qcodeedit-1.0/main.cpp:26: undefined refer
ence to `QFoldPanelCreator::id()'
tmp\obj\main.o(.text+0x1b3):C:/cstuff/qcodeedit-1.0/main.cpp:26: undefined refer
ence to `QLineNumberPanelCreator::id()'
tmp\obj\main.o(.text+0x1d4):C:/cstuff/qcodeedit-1.0/main.cpp:26: undefined refer
ence to `QLineMarkPanelCreator::id()'
collect2: ld returned 1 exit status
mingw32-make[1]: *** [release\example.exe] Error 1
mingw32-make[1]: Leaving directory `C:/cstuff/qcodeedit-1.0'
mingw32-make: *** [release] Error 2

[Matt Smith]

The convention in Qt is that only Trolltech issues classes with names like QClassName. Third-party developers are expected to put initials after the Q to identify it as theirs, as in QfmcCodeEdit. You can find the naming guidelines here. You are supposed to register the initials with Trolltech, but I have registered my own (ij) and have not heard from them and it has not appeared on their site.

[croftj]

Hi,

It would be nice while you are working on such a handy widget is to add a much needed feature which hasn't been done for the last 10 years or so.

It would be nice if you added the ability to do column based selections with copy/cut and paste like Brief (long long time ago) or CRiSP or SlickEdit (more contemporary).

Thanks!

[fullmetalcoder]

It would be nice if you added the ability to do column based selections with copy/cut and paste like Brief (long long time ago) or CRiSP or SlickEdit (more contemporary).

I just added column column-based edit which opens way for selection but I'll need advice on this to ensure that they will not alter the overall usability of the editor...

By the way I did a lot more than column based edit (though it also allows it) : I added something I called "cursor mirrors". These are cursors which mimic the actions of the "real" cursor. You can place them anywhere by holding the Control key while clicking and start typing.

There are some limitations and probably some polishing left to do but it works quite fine and open new perspectives for tedious and repetitive tasks...

[Usability]

I added something I called "cursor mirrors". These are cursors which mimic the actions of the "real" cursor. You can place them anywhere by holding the Control key while clicking and start typing.

Wild! I have never seen anything like that before. It seems likely it will be useful, but only time will tell.

Is the column editing available yet, or just work in progress? I couldn't find any way to select a column yet.

My preference for a user interface is the nedit approach of click then control click to specify the corners of the column, or a control drag to specify its area. The kwrite/kate approach of shift-ctrl-b to toggle between rectangular and normal selection modes always seemed clunky by comparison. But going down the ctrl drag route would mean finding another way to drop your cursor mirrors. Ctrl+alt for the mirrors perhaps.

Incidentally, I did spot a minor bug yesterday while introducing code folding markers into one of our languages. You can reproduce it with the cpp.qnfa file. Try this change

line 38 before
<sequence parenthesis="curly:close" indent="1" fold="1">}</sequence>
after
<sequence parenthesis="curly:close" indent="1" fold="1"> </sequence>

line 38 before
<sequence parenthesis="curly:close" indent="1" fold="1">}</sequence>
after
<sequence parenthesis="curly:close" indent="1" fold="1"> </sequence>

To copy to clipboard, switch view to plain text mode 

Note the close marker is changed from a } to a space. The editor will now segfault on invocation. Sure its a stupid thing do to, and was in fact a mistake, but crashing seemed a bit harsh.

[fullmetalcoder]

Is the column editing available yet, or just work in progress? I couldn't find any way to select a column yet.

Column editing, as far as I understood, is just a restriction of cursor mirrors. I added a way to do it directly : Ctrl + drag the cursor vertically and it will automatically add cursor mirrors all along.

My preference for a user interface is the nedit approach of click then control click to specify the corners of the column, or a control drag to specify its area. The kwrite/kate approach of shift-ctrl-b to toggle between rectangular and normal selection modes always seemed clunky by comparison. But going down the ctrl drag route would mean finding another way to drop your cursor mirrors. Ctrl+alt for the mirrors perhaps.

I was thinking about keeping the "regular" cursor mirrors as they are now and using ctrl+shift + mouse cursor drag to perform column selection... Can you test the current implementation and tell me what you think?

Incidentally, I did spot a minor bug yesterday while introducing code folding markers into one of our languages. You can reproduce it with the cpp.qnfa file. Try this change

line 38 before
<sequence parenthesis="curly:close" indent="1" fold="1">}</sequence>
after
<sequence parenthesis="curly:close" indent="1" fold="1"> </sequence>

line 38 before
<sequence parenthesis="curly:close" indent="1" fold="1">}</sequence>
after
<sequence parenthesis="curly:close" indent="1" fold="1"> </sequence>

To copy to clipboard, switch view to plain text mode 

Note the close marker is changed from a } to a space. The editor will now segfault on invocation. Sure its a stupid thing do to, and was in fact a mistake, but crashing seemed a bit harsh.

There indeed was a caveat in the xml to nfa converter... It made the assumption that all elements add a text child node... thanks for the report, I've just fixed this.

[Usability]

Column editing, as far as I understood, is just a restriction of cursor mirrors. I added a way to do it directly : Ctrl + drag the cursor vertically and it will automatically add cursor mirrors all along.

I have never seen anything like this before, and I think its unique. You might start a trend with this.

However I think the feature requested by croftj is the ability to cut and paste rectangles of text. Most editors have this, just run up kate, kwrite, or nedit and try it. Its ctrl-drag in nedit and ctrl-shift-b then a normal drag in kate/kwrite.

You can cut a rectangle of text and paste it in somewhere else. The cursor mirrors do not help with this.

I was thinking about keeping the "regular" cursor mirrors as they are now and using ctrl+shift + mouse cursor drag to perform column selection... Can you test the current implementation and tell me what you think?

I think both cursor mirrors and column selections have their own uses, and keeping both would be good.

For example if you want to change several lines of "filename.h" to <filename.h>, it would be perfect. However suppose you wanted to pick up characters 4 to 10 of every line, cut them out, and paste them all back in again at column 15. Its simple with a rectangular selection, but I can see no way of doing this with the current qcodeedit. ctrl+shift+drag does the same as a normal drag as far as I can tell, and the cursor mirrors do not appear to help for this.

I think if you add rectangular cut/paste, you would need only one more feature to have an editor that could hold its own with the usual kate/kwrite/nedit crowd. I could get tempted to contribute a simple tabbed GUI for it and use it as a stand alone editor, if as I suspect, qcodeedit2 is faster. Being independent of KDE would be a useful advantage also.

After rectanguar cut/paste the only other important omission would be commenting in/out a block of text. You could convert to comment by prepending what ever the language template identified with context id="comment/single", and convert from comment by removing that.

This is important for languages like VHDL and TCL that only have the type of comment character stops at the end of the line.