custom preprocessor?

[magland]

Is there a nice way to implement my own preprocessor (before standard C++ preprocessing) from within qmake framework.

I envision something like this in the .pro file:

SOURCES += file1.cpp
SPECIAL_SOURCES += file2.cpp
include(process_special_sources.pri)

SOURCES += file1.cpp
SPECIAL_SOURCES += file2.cpp
include(process_special_sources.pri)

To copy to clipboard, switch view to plain text mode 

Here, process_special_sources.pri would somehow preprocess file2.cpp and create special_file2.cpp and add it to the SOURCES variable.

Does this sound feasible? Is there a better way?

Thanks!

[marcel]

Depends on what you want it to preprocess.
Why can't you make use of the compiler preprocessor for replacing macros and other stuff?

[magland]

Depends on what you want it to preprocess.

I want to preprocess source code. Input is source code, and output is modified source code.

Why can't you make use of the compiler preprocessor for replacing macros and other stuff?

Macros aren't flexible enough to do what I want to do. I want to have an external program do custom preprocessing.

Any ideas?

[marcel]

Then you want to do something similar to what moc does.
Sounds a lot of work. Maybe you can reuse some pars of moc.

[magland]

Then you want to do something similar to what moc does.
Sounds a lot of work. Maybe you can reuse some pars of moc.

Yes, you are right. I want to do something similar to moc. However, the difference is that moc reads the source files and creates new (additional) source files. Whereas I need to modify the source files in place. Aaccording to wysota, Q_OBJECT, signals, slots, etc. are just macros (although they do give information to moc), so there's really no preprocessing that takes place by moc, in the sense that I want.

[marcel]

There is no ready functionality to do what you want. That is why I suggested you should reuse some moc parts, such as the code parsing.

I am aware that you want top modify the same source file, not generate a new one, but as I said, you have to create a new tool to do that. It is better to reuse moc functionality since it clearly works, and you won't have to write new things and test them.

[magland]

Okay, thanks marcel. I'll look into that. In the meantime if anyone has experience with setting up this type of thing, please let me know.

[magland]

For information of those who are curious.

I got the custom preprocessor to work very nicely using only the following code in .pri file!

for(file, CHAINLINK_SOURCES) {
	system($$(CHAINLINK_DIR)\bin\cpp_preprocessor $${file} CHAINLINK_$${file})
	SOURCES += CHAINLINK_$${file}
}

for(file, CHAINLINK_SOURCES) {
	system($$(CHAINLINK_DIR)\bin\cpp_preprocessor $${file} CHAINLINK_$${file})
	SOURCES += CHAINLINK_$${file}
}

To copy to clipboard, switch view to plain text mode 

Then when preprocessing the file I keep the line numbers updated using the #line macro... so that to the user it's seamless, i.e. when there is an error in the source file, the compiler output points to the original source (not the pre-processed source).

Really quite a simple system... but it opens huge possibilities (which I need to be careful not to abuse )

Any comments, suggestions?

[marcel]

OK, nice, but you should really have mentioned what exactly you need to preprocess .
I initially thought you had some code+macro constructs that you needed to be expanded to something by a 3rd party preprocessor.

[magland]

OK, nice, but you should really have mentioned what exactly you need to preprocess .
I initially thought you had some code+macro constructs that you needed to be expanded to something by a 3rd party preprocessor.

Yes, sorry about that... I didn't want to confuse the issue before the specific problem was solved. Here's what I can do now (and I'll release this software sometime soon). ~~ very exciting ~~

//I include this single header /////////////////////
//and link to chainlink_engine library ///////////
#include "chainlink_engine.h" 
 
int main(int argc, char *argv[]) {
 
QString str1=argv[1];
 
CHAINLINK(QString str1) {
  %in this block I execute arbitrary chainlink (Matlab) commands!
  X=1:1000;
  Y=sqrt(1:1000);
  H=plot(X,Y);
  setWindowTitle(H,str1); %I can even use variables passed in from C++
  str1=str1+' hello world'; %and even modify these!
}
 
cout << str1; //and see the result in C++
 
//now we are getting somewhere :)
 
return 0;
 
}

//I include this single header /////////////////////
//and link to chainlink_engine library ///////////
#include "chainlink_engine.h" 

int main(int argc, char *argv[]) {

QString str1=argv[1];

CHAINLINK(QString str1) {
  %in this block I execute arbitrary chainlink (Matlab) commands!
  X=1:1000;
  Y=sqrt(1:1000);
  H=plot(X,Y);
  setWindowTitle(H,str1); %I can even use variables passed in from C++
  str1=str1+' hello world'; %and even modify these!
}

cout << str1; //and see the result in C++

//now we are getting somewhere :)

return 0;

}

To copy to clipboard, switch view to plain text mode 

[mchara]

Maybe a little cheating - name your application moc.exe, rename original moc as i.e. Qtmoc,
do your tasks in your moc and call qtmoc from inside...

[magland]

Maybe a little cheating - name your application moc.exe, rename original moc as i.e. Qtmoc,
do your tasks in your moc and call qtmoc from inside...

Thanks, but the problem seems to be solved... without cheating

[fullmetalcoder]

Maybe a little cheating - name your application moc.exe, rename original moc as i.e. Qtmoc,
do your tasks in your moc and call qtmoc from inside...

Man that wouldn't make any sense... If you ever had a look at all the files lying in $QTDIR/mkspecs you'd realize how bad this method is... The thing is, all subcommands (uic, rcc, moc, ...) are not supported directly by qmake but through .prf files which "registers" them as "extra compilers".

An extra compiler basically takes all the file from a given variable, calls a set of command, passing these files (one by one) as parameters and is able to place new files (result of its processing) into some variables (SOURCES for example). Other niceties such as deps tracking are also allowed through this construct. all you need is to have a look at your mkspecs directory. In the features subfolder you'll find a couple of interesting files : uic.prf, moc.prf and rcc.prf (or resources.prf, I don't remember...). As a an alternative you might consider reading the one I created to manage some plugin-oriented processing in Edyuk ...

Besides, the advantage of using a prf file is that once you install it to the above mentionned place, any project can use it as follow (this is just an example... In my case "chainlink_preprocessor" is actually "qplugin_from_scheme" but I guess you understood ):

CONFIG += chainlink_preprocessor

Hope this helps.

[magland]

Thanks fmc, that's good info. But I really need something that I can distribute easily in a release... something as simple as a .pri file to be included at the bottom of the project file (as I described in previous post) does the job perfectly.

I've got the system working really well now, and I can include ChainLink scripts seamlessly in the C++ code like this:

#include "chainlink_engine.h"
int main(int argc, char *argv[]) {
  QApplication app(argc, argv);
 
  integer my_int=5; //I declare some data in C++
 
  //I can pass this variable into chainlink, as long as chainlink recognizes the data type
  CHAINLINK << integer my_int 
  CHAINLINK {
    X=rand(100,100,my_in); %Now I can excecute arbitrary chainlink commands here
    H=viewmda(X);
  }
  QWidget *H;
  //and then even retrieve a pointer to the widget created in chainlink
  CHAINLINK >> QWidgetPtr H
  if (!H) return -1;
  H->setWindowTitle("Hello World!");
  return app.exec();
}

#include "chainlink_engine.h"
int main(int argc, char *argv[]) {
  QApplication app(argc, argv);
 
  integer my_int=5; //I declare some data in C++
  
  //I can pass this variable into chainlink, as long as chainlink recognizes the data type
  CHAINLINK << integer my_int 
  CHAINLINK {
    X=rand(100,100,my_in); %Now I can excecute arbitrary chainlink commands here
    H=viewmda(X);
  }
  QWidget *H;
  //and then even retrieve a pointer to the widget created in chainlink
  CHAINLINK >> QWidgetPtr H
  if (!H) return -1;
  H->setWindowTitle("Hello World!");
  return app.exec();
}

To copy to clipboard, switch view to plain text mode 

Only one extra line needed at bottom of project file!

[fullmetalcoder]

Thanks fmc, that's good info. But I really need something that I can distribute easily in a release... something as simple as a .pri file to be included at the bottom of the project file (as I described in previous post) does the job perfectly.

A .pri file is not simpler than a prf .... Indeed, it forces the coder to add manage one more file and to make sure it is compatible with the "backend"... Using a prf "globally" (i.e. installed at the same time your preprocessor is, and in a standard location make it simpler for someone to use your framework without having to bother about potential internal breaks (for example in the CLI of your preprocessor...). And as I said already, it is more natural to most Qt users to add an extra config feature than to rely on the include() directive.

[magland]

A .pri file is not simpler than a prf .... Indeed, it forces the coder to add manage one more file and to make sure it is compatible with the "backend"... Using a prf "globally" (i.e. installed at the same time your preprocessor is, and in a standard location make it simpler for someone to use your framework without having to bother about potential internal breaks (for example in the CLI of your preprocessor...). And as I said already, it is more natural to most Qt users to add an extra config feature than to rely on the include() directive.

The way I have it set up, they would install chainlink and set the CHAINLINK_DIR environment variable on their system. Then they just need to add a single include statement at bottom of their .pro file:

include($$(CHAINLINK_DIR)/chainlink_engine.pri)

include($$(CHAINLINK_DIR)/chainlink_engine.pri)

To copy to clipboard, switch view to plain text mode 

I agree that a configuration option is more natural... however there are two important advantages of the .pri system in my situation:
1. If they upgrade Qt they will need to reinstall the prf (whereas with current system it will just work.
2. If they desire to see what's going on behind the scenes (because they are programmers after all), it will be easy to follow the "include" statement. After all, the .pri is only four lines (see a previous post)!

[fullmetalcoder]

1. If they upgrade Qt they will need to reinstall the prf (whereas with current system it will just work.

That's not necessarily true... It actually depends on how Qt get installed... For instance, there would be no such trouble under Linux distro that ship Qt packages or if you overwrite your previous Qt installation when upgrading (whathever the system)...

2. If they desire to see what's going on behind the scenes (because they are programmers after all), it will be easy to follow the "include" statement. After all, the .pri is only four lines (see a previous post)!

They could still see what happens behind the scene by having a look at the prf packaged with your program (not the one installed).

Anyway, the choice is up to you... I just wanted to let you know about the prf-based alternative

[magland]

Anyway, the choice is up to you... I just wanted to let you know about the prf-based alternative

Yes, I understand, and appreciate that... I may look into it for the future.

I think custom preprocessors can be a very powerful tool if used in the right away. On the other hand there is a risk of abusing them. I am very impressed with how the Trolls handle it... they have a simple system which is sufficient, and they stick to it. You know, they could add all kinds of tricky stuff to the Q_OBJECT, moc, etc system, but I am confident that they will think it out very carefully before they do such a thing.