Python preprocessor script for #includes and compiler selection

J G

Active member
Hello,
This is my first post, so please let me know if I have done something wrong in regards to the forum.
I recently made my laptop dual boot Windows 10 and Ubuntu and have found that the picaxe preprocessor has been one of the few remaining things requiring me to boot into Windows due to my latest project having several linked files using #include "filename.basinc".
This thread in the forums outlined the situation: https://picaxeforum.co.uk/threads/macros-with-axepad.29809/

To partially work around this, I have written a python script that will process #include, #com and #picaxe directives and start one of the picaxe command line compilers from here: https://picaxe.com/software/drivers/picaxe-compilers/
When given a picaxe basic file, it will create a file with the given and all included files merged into one, then calls the correct compiler to compile or upload the code to the picaxe. It will also process the #picaxe and #com directives to select the compiler to use and serial port (although I think that the compiler can use the #com anyway). The picaxe chip can also be selected as an argument when calling the script through the use of -vPART.

All other command line arguments attempt to emulate those used by the compilers (-s, -f, -c...) and are passed onto them (with the exception of the serial port, which may be overwritten by any #com in the files. In terms of usage, hopefully it is fairly similar to using the compilers directly.
For example, an 18m2 could be programmed as follows:
Code:
picaxe.py -v18m2 -c/dev/ttyUSB0 test.bas
Currently, macros, #DEFINE, #IF and #ELSEIF are not processed and are left exactly as they are, which may or may not cause the compiler to misbehave if they are used (I haven't tested them as I have not used them too much myself).

The script and a simple set of picaxe files to test with can be found here: https://github.com/jgOhYeah/PICAXE-Libraries-Extras

This preprocessor is aimed at linux, however I don't think there is anything particularly platform specific, so it may work on other things as long as the filepaths and names of the compilers are set correctly. I am not sure if it is the correct place to locate them, but I copied the script into /usr/local/bin and the compilers to /usr/local/lib/picaxe/picaxe****, where picaxe**** is each compiler file. This means that I can access it anywhere on the system without needing to remember where the script is saved.

There may be bugs and there definitely will be things to improve on, so feedback and or improvements are welcome.
 

pleiser

Senior Member
Welcome to the PICAXE forum, and thanks for sharing your project!

A couple years ago, I actually made a very similar project, coincidentally also using python. It's at https://github.com/Patronics/PicaxePreprocess, and, like yours, also has some sample files and examples included.

It seems our projects have some, but not all features overlapping (I implemented #include, #define (Specifically the inline-macro portion of #define's functionality, as AxePad does the rest), and #macro). In my case it's configured for use with Linux and MacOS, and also attempts to auto-detect the USB cable.

Depending on how similar the implementation is, perhaps you'd be interested in adding some of the features from my version to yours, or contributing your features to mine. Regardless, I figured it'd be helpful to share a link to mine in case it's helpful to anyone who's looking for something like this.

Edit: I just added a link to your preprocessor in my readme, in case it's helpful to anyone else who finds mine.
 
Last edited:
  • Like
Reactions: J G

J G

Active member
Ok. Good to know that there are others out there and thanks for sharing it. I have to admit that I didn't go looking too hard for any previous / existing ones as I thought it would be a fun challenge to start from scratch :)

When I get time, I might copy the chip selection and starting the compiler over to yours as selecting and starting the correct compiler is a separate step in mine that could be relatively easily added as long as the chip and port are specified or are found in the file. Sending to the compiler would be enabled / disabled with another option such as -u for upload if that is alright.

The actual merging parts of both look quite similar in structure, although was yours written for python 2? I added brackets around the print statements in my local copy and it seems to work alright with python3 though, so hopefully no issues converting between python versions apart from that.

In the mean time I have also added a link to your preprocessor in the readme of mine to help people who might be looking.
 

pleiser

Senior Member
Sorry for the late reply, college finals have been keeping me busy!
Ok. Good to know that there are others out there and thanks for sharing it. I have to admit that I didn't go looking too hard for any previous / existing ones as I thought it would be a fun challenge to start from scratch :)
Yeah, same for me, I later found a C preprocessor (also linked in my readme) that would have probably done enough, but where's the fun in that?
When I get time, I might copy the chip selection and starting the compiler over to yours as selecting and starting the correct compiler is a separate step in mine that could be relatively easily added as long as the chip and port are specified or are found in the file. Sending to the compiler would be enabled / disabled with another option such as -u for upload if that is alright.
Sure! Feel free to free to add any additional options or features you want, I definitely appreciate any and all contributions!
The actual merging parts of both look quite similar in structure, although was yours written for python 2? I added brackets around the print statements in my local copy and it seems to work alright with python3 though, so hopefully no issues converting between python versions apart from that.
Yeah, at the time I wrote it I had more experience with python 2, I can go through and update it. (If you already made the changes successfully though, a pull request is welcome!)
In the mean time I have also added a link to your preprocessor in the readme of mine to help people who might be looking.
Hmm, are you sure you committed and pushed this change? I can't find it on your Github readme.
 
  • Like
Reactions: J G

J G

Active member
Hmm, are you sure you committed and pushed this change? I can't find it on your Github readme.
Thanks for that. First time using command line git to push directly to github, so must have entered an incorrect username or password or something :)

I have copied the various extra features from mine into your code and it seems to be mostly working, although I think I have broken some of the filepath handling, so I will fix that and then submit a pull request once I remember / work out how to.
 

J G

Active member
I have submitted a pull request, so hopefully it works correctly and you get it.

Feel free to try and break it / test it as I have only tested it in situations I would commonly use it with or know about and with a limited set of files, so there may be issues that have not been found yet, especially with filepaths and working directories, but it seems ok for me.
 

J G

Active member
Hello,
I have been playing around with getting #IF, #IFDEF, #IFNDEF, #ELSE, #ELSEIF, #ELSEIFDEF, #ELSEIFNDEF (I know this isn't in the manual, but it was easy to implement), #ENDIF and #UNDEF to work and have submitted a pull request. No pressure or rush to get it approved - it is a busy time of year :).

It might need a little bit of testing to make sure it isn't doing anything too stupid, but it appears to be behaving for me.

As an aside, while testing, I think I have come accross a few minor inconsistencies in the way PE6 handles some preprocessor directives, with #UNDEF not seeming to be implemented even though it is in the manual and some #IFs being partially evaluated inside block comments made using the #REM and #ENDREM directives. Has anyone else had the same experience? I am not too sure what version I am running though and it is not big issue for me as I rarely use undefines with any preprocessor, but interesting to note.

Have a safe and happy new year.
 

pleiser

Senior Member
I finally got around to merging in your changes, (sorry I didn't get around to it sooner), thanks again for your contributions! I also did some more testing with this version, and realized (and fixed) an issue with accounting for the fact that the PICAXE 40x2 uses the same compiler as the 28x2. I also decided to add a few more features, so now it also supports the official preprocessor substitutions, which I've never used before (are they new to PE6?), but I figured could be useful.

I'm hoping to soon finally make the jump in updating to macOS Catalina, so I needed to ensure that the preprocessor (along with the 64 bit command-line compilers) are ready to take over the role of AxePad, and it does seem to be sufficient now! Also, to reduce the potential problems that could be caused if we do inadvertently make a breaking change, I've started implementing releases.
 
  • Like
Reactions: J G

J G

Active member
That looks good. Good pick on the 40x2 compilers and thanks for adding ppp* substitutions and merging everything together.

I would have to check how PE6 handles it, but would it be possible to add the ppp* substitutions as a define when creating the defines dictionary or using the replace() function in the file rather than the str.replace() method as either of these would not substitute when they shouldn't, for example in strings and comments? As an example:
Code:
sertxd("ppp_date_uk is ", ppp_date_uk)
should output:
Code:
ppp_date_uk is 24/02/2021
Not sure if that would ever be useful or is correct to PE6, but that is how I would expect things to behave :) At one point I had started implementing ppp* as a predefined define, but hadn't ever gotten that far with it. I am happy to modify your implementation and create a pull request or feel free to do (or not do :)) it yourself.

I like the idea of having releases. Makes it easier to recover in the mean time if something does get missed in testing.

As an example for using ppp* substitutions, in recent projects that might not be touched very often once installed, I have started putting a string that prints a short description of the device and the date the code was compiled to give me an idea of what code is currently running on the device, what it thinks it is and how long since the code was last touched whenever I have to do something to the device next. There is also a thread around that describes using the date as a serial number to identify picaxe chips on a network.
 
Top