Wonky memory used numbers in 20X2 compiler

amdunn

New Member
I've got a fairly large program that I'm downloading into a 20X2 using the command-line Linux compiler, version 3.1



If I download the program with a small amount of code commented out, I get the following:

DOWNLOADING R01.bas

PICAXE-20X2 Compiler
Version 3.1
Copyright (c) 1996-2014
Revolution Education Ltd

Compiled successfully.
Memory used = 3890 out of 4096 bytes.



This suggests I have 206 bytes free.

If I enable one line of code more, I get the following:



DOWNLOADING R01.bas

PICAXE-20X2 Compiler
Version 3.1
Copyright (c) 1996-2014
Revolution Education Ltd

^
line# 1877, col# 246

Error: Memory full - -255 bytes over. Remove the 'table' command to release 256 more bytes or reduce program length by -255 bytes.






The line I commented in is the one shown below in bold (b0-b0*10). With that line commented out, it compiles with 200+ free bytes. With that line in, I get the error above.


;
b47=222
; AMD
read 15,b0 ; get app lt config value
if b0 < 1 then nappl
b0=b0*10 ; app lt config value in 10s of seconds max 9
; b47=b0
;






The math here doesn't add up. If I really had 206 bytes free before, there's no way that one statement "b0=b0*10" used them all up.

Is this an issue with the compiler (suspected because of the negative number of bytes that would be "freed up" per the error message)?
 

hippy

Technical Support
Staff member
There have always been anomalies in the way the number of bytes are reported where code is counted but not data, where what's used may differ from the space that will take up or the total available on devices with shared memory and I guess this would be an example of that.

What can happen is that an extra line can push things over a boundary, cause the block it falls in to be marked as used, which means a whole lot more is then used than the amount pushed over the boundary. Then things get complicated when it comes to how much more would actually be needed.
 

amdunn

New Member
That's a pretty big "anomaly"... to suddenly lose your last 200 bytes (that you thought you had) of program memory is concerning!

Is there a way to confirm that's really what is going on... ie. that it isn't a compiler bug or something similar? If it's really true, there's a LOT of code rewriting I'm going to have to do to squeeze things in, drop features, etc. and I'd hate to do all that major surgery and then find out there really was more room available if only I'd done 'xxxx'.
 

hippy

Technical Support
Staff member
I can't think of a way to prove it's not a compiler bug but would not expect it to be. Given "line# 1877, col# 246", 3890 bytes used before the addition, that would suggest you have reached the limits of memory with TABLE being used.

There are 4096-256 (3840) bytes of guaranteed code space, the 256 bytes can be code or table data. I would guess you have used 50 bytes of that dual-use / shared memory for code but using more encroaches on what is used in the table.

You could temporarily remove TABLE and READTABLE commands which should allow you to add more code, demonstrating it is such an issue.
 

Buzby

Senior Member
I've not got a Linux environment to test with, only Windows and PE6

The compiler in PE6 is at v4.0, yours shows v3.1.

I tried lots of different ways of going over memory, using both table and no table.

The column number in the 'memory full' error message was always 0.
Where did your #246 come from ?

Can you switch on the 'view pre-processor output' option like the one in PE6 ?
 

amdunn

New Member
The program uses exactly 68 bytes of TABLE... and I would assume (although one shouldn't assume) that those bytes are calculated in the "Memory used = 3890 out of 4096 bytes" in the first (working) example, since the exact same TABLE statements are in both cases.

So I guess the answer may lie in how the shared / dual-use memory is allocated... I've seen nothing to suggest it's other than individual bytes, but maybe it isn't... but if it is allocated in blocks, I can't begin to guess at an algorithm that would explain the capacity numbers/messaging in my original post.

Deleting the TABLE statements does, in fact, make the error go away. So it looks like it is all or nothing on TABLE, but the math is still wrong.

With the TABLE statement in place, but the one offending extra line of code commented out, the compiler tells me I have used 3890 out of 4096 bytes. If the TABLE statement really allocates all 256 bytes, then you'd have only 3840 bytes available... so how could I have used 3890 of those 3840 bytes?

Or if the 3890 is meant to include TABLE, maybe it only includes the 68 bytes of active TABLE space, although the remaining unused 188 bytes of the 256 byte table space get marooned somehow. That's closer to making the math work, but still not quite.

If I comment out the line I originally referred to, it frees up 7 bytes of program memory. So that's how much space the "b0=b0*10" statement consumes when compiled. That's believable.

Maybe that's the math... the compiler message "3890" bytes is 68 bytes of TABLE and 3822 bytes of program code. That's closer. When I then add the "b0=b0*10" statement it uses 7 more bytes, pushing us up to 3829 bytes of program code (of the 3840 we think is available). The math is still wrong... but closer. Maybe the program code is allocated in blocks as well.

If that's true... the misleading thing is the compiler message saying "3890" when the real answer is probably "3890+188" (counting the marooned space that's allocated to table - all 256 bytes of it - but I haven't included TABLE statements for 188 of those bytes) or 4078 out of 4096 bytes. That, at least, would have given me a hint I was much closer to the limit. Adding the 7 byte statement in then pushes me to 4085 and at 4085 out of 4096 I'd have understood how close I was getting.

The interaction between the various storage types is also not well documented. For instance, in the FAQs it says this:
"The PICAXE X2 parts have 4096 bytes of program and 256 bytes of EEPROM data memory. ... The PICAXE X2 parts include 256 bytes of table data, similar to EEPROM data memory but read-only." There are hints that there is an interaction between the TABLE space and the program space. That is documented on the TABLE command but not documented in the FAQ... so the assumption is that the FAQ is wrong and the TABLE command is right.
 

amdunn

New Member
I've not got a Linux environment to test with, only Windows and PE6

The compiler in PE6 is at v4.0, yours shows v3.1. I tried lots of different ways of going over memory, using both table and no table.
The column number in the 'memory full' error message was always 0. Where did your #246 come from ? Can you switch on the 'view pre-processor output' option like the one in PE6 ?
Not using the PE because I only have a Linux environment... opposite of your situation. So I don't know what the PE would show.
 

Buzby

Senior Member
Can you use 20X2 compiler v4.0 in Linux ?

The column number in your error message looks weird. #246 would be a very long line !

This is the sort of error message compiler v4.0 gives ...

PICAXE-20X2 Compiler
Version 4.0
Copyright (c) 1996-2017
Revolution Education Ltd

^
line# 119, col# 0

Error: Memory full - -254 bytes over. Remove the 'table' command to release 256 more bytes or reduce program length by -254 bytes.


The column number is always zero.
 

hippy

Technical Support
Staff member
This is my diagram which covers how the memory layout is -

MEM-ACTUAL.jpg
Click to enlarge

The latest compilers for Linux are 3.1 versions. I don't believe there is anything different between those and later Windows versions when it comes to reporting memory usage.
 
Last edited:

amdunn

New Member
This is my diagram which covers how the memory layout is -

View attachment 25326
Click to enlarge

The latest compilers for Linux are 3.1 versions. I don't believe there is anything different between those and later Windows versions when it comes to reporting memory usage.
Yes I very much like that diagram... it covers the difference in layout between the families of parts.

But here's the part causing confusion. In the X2 diagram above, it LOOKS LIKE there is a variable amount allocated to the table (diagonal line), and that variable amount is deducted from code. I think that's what most people believe happens with X2 parts... but the numbers above in this thread disprove that interpretation. The numbers prove that what really happens is that if you use even 1 byte of table, then all 256 bytes are taken away from code. The compiler reports those "missing" bytes as available, rather than used. That's why I'm getting the case where I add a single line of code and when I thought I had over 200 bytes of code space free, I now have none.

In fact, I can prove that.

With no TABLE statements at all: Memory used = 3838 out of 4096 bytes.

Add a single byte of TABLE: Error: Memory full - -244 bytes over

Remove a small amount of code, but keep the single byte of TABLE: Memory used = 3823 out of 4096 bytes.


The bug is that the compiler reports the "allocated" space you didn't use in the TABLE (in the example I just wrote, 255 bytes) as unused, leading the user to believe it is available for code, when it isn't.
 

hippy

Technical Support
Staff member
But here's the part causing confusion. In the X2 diagram above, it LOOKS LIKE there is a variable amount allocated to the table (diagonal line), and that variable amount is deducted from code. I think that's what most people believe happens with X2 parts... but the numbers above in this thread disprove that interpretation. The numbers prove that what really happens is that if you use even 1 byte of table, then all 256 bytes are taken away from code.
Yes it appears you are correct; use one byte and all 256 are allocated as table, and the image doesn't actually reflect that. I have updated the image.
 

amdunn

New Member
How we got started on this is that the compiler also doesn't report that correctly, or at least not in a way that's meaningful to the programmer.

It appears the compiler only reports memory used for TABLE when you actually include those bytes in a TABLE statement. So in the example where you add a 1 byte table, until you actually run out of memory, the compiler will count that as 1 byte used, not 256 -- although the other 255 are no longer available to the programmer. Then you run out of code space 255 bytes sooner than you thought you would... memory used will jump suddenly from 3840 out of 4096, to MEMORY FULL.

Yes, I suppose the 255 bytes of not-used table are 'technically' not used... but they are only available for the programmer to expand the table, and cannot be used for code.

In reality there are three numbers - actual code used, actual table used, and memory allocated to as-yet-unused possible table entries.

I guess the proper answer might be something like this:

Memory used = 3822 (code) + 68 (table) + 188 (allocated to table) out of 4096 bytes.

This gives the correct impression (that I was lacking at the start) that there are only 18 bytes more available for code, and adding a couple of lines of BASIC will push you over the limit. Otherwise it looks like you have 200+ bytes to go. Big difference between 200 free and 18 free!

I'm sure others have been misled by this as well.
 
Top