Calculating integers far greater than 65535.

Jim-TX

New Member
If I am not mistaken... the numbers that you are obtaining from the ADC - are... not ohm values... but a voltage value that then needs to be converted to a resistance value?? (Showing my ignorance here...)
You have a voltage divider circuit. +5 volts across the 1k and the 'X' resistor.
 
Last edited:

mortifyu

New Member
I can't get your readings from Post #30 to correlate with the resistances you are reading ...
Yes, they don’t add up to me either. Give me a few days and I will post again specifically providing new figures all round. ie Exact R values ADC’s etc.

Regards,
Mort.
 

hippy

Technical Support
Staff member
For a schematic, see #30 above
Oops - Seems I missed that. Thanks. So, in essence -
Code:
Vref -.--------------.---
     .|.             |
  R1 |_| 1K          |
      |       Vadc   |
      }--------.     |
      |       .|.    |
      |   10K |_|    |
      |        |  ___|___
     .|.      _|_/       |
  R2 |_|     |_ _   ADC  |
      |        | \_______|
      |       .|.    |
      |   10K |_|    |
      |        |     |
  0V -^--------^-----^---
From that we can calculate what Vadc should be ( ignoring any effect of the two 10K's and the effective resistance of the ADC input ) as -

Vadc = Vref * R1 / ( R1 + R2 )

And we can determine what the 30-bit result of the ADC reading should be from -

Nadc = $3FFFFFFF * Vadc / Vref

Which gives us ...
Code:
           ----- Expected -----   ------ Actual ------
R2 = 0.1      107363  $0001A363      392377  $0005FCB9
R2 = 330   266417144  $0FE133F8   533314220  $1FC9BAAC
So something is 'way out of whack' there.

The other concerns I have are that, with the two 10K's and effective resietance of the ADC, the R2 being read won't be R2 but something slightly less, which will reduce Vadc by some amount. In addition, because of the 10K's, the +Input of the ADC won't be Vadc but something less, and -Input won't be 0V but something higher. That will reduce what the Nadc reading is against what it would be in an ideal world.

I guess none of that really matters as the ADC reading can still be reversed. But it will make that reversal much more complicated if that needs to be taken into account.

I'm not sure why either 10K is there and I think it would be worth removing both if there's no good reason for their presence.
 

mortifyu

New Member
If I am not mistaken... the numbers that you are obtaining from the ADC - are... not ohm values... but a voltage value that then needs to be converted to a resistance value?? (Showing my ignorance here...)
You have a voltage divider circuit. +5 volts across the 1k and the 'X' resistor.
The ADC values are essentially a ratio metric digital representation of a voltage. This particular project will only ever have a MAX R2 resistance of 330 ohms. Just to provide you an appreciation of this circuit/setup, theoretically a MAX decimal value of 24-bits is 16,777,215 (a far cry above PICAXE 10bit ADC - MAX 65536) which in this case would be 5.000VDC. In a ‘perfect world’ if R2 was also 1K exactly, the ADC value would be 8,388,607 and the voltage would be 2.500VDC.

@hippy - If you worked your MATH with these references, hopefully you’d be able to come up with the mathematical result to 0.1 ohm resolution. As we know, the above quoted values are absolute.

Perhaps to keep the resultant MATH to three digits and one decimal place, you could work with the absolute known values For R2=500.0 ohms. That is, ADC value = 5,592,405 and Vout would = 1.6VDC recurring.

Regards,
Mort.
 

mortifyu

New Member
Yes, sorry Hippy, I am pretty sure I have given you a bum steer with earlier values and took them without factoring in slight inaccuracies in R1=1K and R2 being 329.2 ohms measured. But, if you work with the theoretically accurate known values noted earlier, perhaps we/you can get to where I am lost trying to get PICAXE to solve the MATH.

Regards,
Mort.
 

mortifyu

New Member
And we can determine what the 30-bit...
It is a 24bit value we are working with.

Bit 5 of first byte through to bit 6 of 4th byte.

The pair of 10k resistors at the ADC are not at all significant in the circuit or equations, but do need to remain there for protective purposes.

Also, if anyone is interested (please speak up) I am happy to post the GERBER PCB files (not that I have generated them just yet) for this circuit so you could email them to a board house of choice and have the suitable PCB made. It makes no difference to the calculations, but there is also an inductor on the output from the 5.000VDC REF supply that is not shown in the circuit as in the end this will be a printed PCB track inductor most likely.


Regards,
Mort.
 

Jim-TX

New Member
The numbers don't make sense.
If 16,777,215 = 5.0V and 8,388,607 = 2.5V, then it stands to reason that 4,194,303 = 1.25V.
When you have a 330 ohm resistor plugged in as R2, you are getting a reading of 8,333,034. According to the above numbers, that is very near 2.5V.

But - if you do the math on a 1k and 330 ohm in series, there should be 1.24V on R2. This should put the reading near to 4,194,303.

The same goes with the 0.1 ohm resistor. The voltage across a 0.1 ohm resistor should be somewhere near 0.0006V. This means a reading should be near 2048. You are reading 6130 which is about 0.0018V (2048 + 4095). This doesn't make sense.

If it were me, I'd use a precision 330 ohm resistor for R1. Then the voltage across R2 should be very near 2.5 volts when you have a 330 plugged in as R2.

This would also make the voltage across R2 when it is a 0.1 ohm resistor higher... 0.0015V, which gets it up higher out of the noise.

The other thing I would do is to place a bus wire in the circuit to be measured - instead of a 330 ohm or 0.1 ohm. Get a baseline reading. Theoretically it should be 0 - but there will be some resistance in the bus wire, so you will have a number. Regardless, that should be your 'zero' volts.

Then plug in a high value resistor - say, like 300k. 300k + 330 = 300,330 ohms. The voltage across the 300k resistor will be something like 4.994 volts. Not exactly 5.0 volts, but close enough to give you a reading of what it should be near the top end.

Now you have some real-world numbers to work with.

From what I can see, you need to get some real data. Some real numbers that make sense before you go off working on any math equations.

But… that's just me.
 

mortifyu

New Member
The theoreticals in a ‘perfect world’

0.1ohms = ADC 1118 or %10001011110 or $45E and measures 0.00025VDC or 250uV

330.0 ohms = ADC 3,689,400 or %1110000100101110111000 or $384BB8 and measures 1.10VDC recurring.


The resolution Is easily available to achieve 0.1 ohm steps.

With R1 being 0.01% (+/- 0.1 ohm) tolerance, theoretically at worst the overall accuracy of this circuit/setup being +/- 0.1 ohms is going to come down to just how stable I can get the 5.000VDC reference.


Regards,
 

mortifyu

New Member
...When you have a 330 ohm resistor plugged in as R2, you are getting a reading of 8,333,034.
As previously posted, the earlier values reported were incorrect with a number of factors not being accounted for. Since then, this circuit/setup and tolerances have been dramatically improved.

The suggestion of R1 also being 330 ohms is valid, however this then presents an unacceptable current problem. R1 in this particular case must NOT be under 1.0k ohms.

By PCB design and accurately calculated filtering techniques supplying the 5.000VDC reference, less than 150uV noise should be achievable.

The only other thing that may also need to occur to assist in achieving this level of stability during the actual ADC read is a momentary halt of all local clock oscillators other than the ADC chip itself.

Regards,
Mort.
 

Jim-TX

New Member
Sooo… I guess I am still trying to figure out what you are reading. Your actual number that the ADC is giving you. The 32-bit - yes, I know it needs to be massaged to 24-bits - but what is the actual 32-bit binary number that you are getting - for both resistors.

330 ohms = xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx
0.1 ohms = xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx

There is still a disconnect between what you are now posting (#49) and what I seem to recall from Ohms Law.
 

mortifyu

New Member
I can't provide ACTUAL's atm. This is because the PCB has not been manufactured yet. To provide relative 'accurate' actual reads, I need to print and build this circuit to a PCB. Currently it is in 'better than usual' breadboard form, but still not at all good enough to provide a good ADC read (Such as we have already seen the rather poor reads I provided early on).

To print the PCB, I first want to be 100% confident the MATH can be performed by a PICAXE. Otherwise what is the point.

I hear what you are saying reference to ADC calcs VS Ohms Law.

Unusually I have now put this dilemma to a couple of professors I know and am waiting for their responses to which will be posted here.

I am actually also quite eager to hear/see/read AllyCat chime in here now.



Regards,
Mort.
 

Jim-TX

New Member
Oh. Okay. Well, in all of this conversation, I was working on a method for determining the voltage - based upon the raw values read.

If I were doing it, I would use a SELECT CASE statement. But that's just me. I'm sure you will get a much better response from the experts here.
 

marks

Senior Member
Hi Mort,
I dont think your that far off the mark.
for 0.12 ohms you should be getting a value of 4026
and 329.2 ohms would be 8310351 (I think what Jim-TX was trying to elude to you should be able to measure 1.2383v across R2)
calculating your result of 8333034 (would suggest you have 1.2417v across R2) which calculates to 330.39 ohms

for commonmode to get your 24 bit result you need to divide by 2
so 8333034 is actually 4166517 / 16777215 x 5.00vRef = 1.2417v
 

mortifyu

New Member
Hi marks,

Want to have a go explaining working down to 330 ohms from a MAX 24bit value being equal to 16,777,215 decimal?

Regards,
Mort.
 

marks

Senior Member
ADCvalue = R2 / (R1 +R2) X 16777215
= 330 / (1000 + 330) X 16777215
= 4162767

R2 = ADCvalue X R1 / (16777215 - ADCvalue)
= 4162767 X 1000 / (12614448)
= 329.9999
 

AllyCat

Senior Member
Hi,

Ah well, as my name was mentioned, IMHO the calculations really aren't very difficult, but the numbers keep changing and, like hippy, I missed the change in the resistor value to 1k ohms.. Firstly, you don't NEED even 24-bit measurements, 16-bits (Words) should be more than enough. You only need to use higher values (e.g. 32 bits) for "intermediate" results or incorrectly scaled values (i.e. too small or too large). Personally, I think the data in #30 is in the most "useful" form to demonstrate:

R2 @ 0.1 ohms (measured 0.12 ohms)
b1=128, b2=5, b3=252, b4=185 or in binary, 10000000 00000101 11111100 10111001

R2 @ 330 ohms (measured 329.2 ohms)
b1=159, b2=201, b3=186, b4=172 or in binary, 10011111 11001001 10111010 10101100
For now (and maybe forever) we can work with just the High Words, consider those 32-bit numbers split into an "integer" word and a "fractional" word (which we can ignore for now). Ignoring also the two top "Status" bits, the value for the High word for 330 ohms is 31 * 256 + 201 = 8137. The total divider resistance is 329.2 + 1000 = 1329.2 ohms, so the value corresponding to the "ADC Reference" value is 8137 * 1329.2 / 329.2 = 32854. That's quite "encouraging" as it's just 0.26% above the well-known 32767 which is the largest positive number possible in a 16-bit two's-complement (signed) number. But it is an "overflow" of 87, which would have been higher if we'd included the "fractional" part of the original number. So the main error is in the "measurement" not the calculation.

Doing the same with the "0.1 ohm" value we start with a value of only 5, but the fractional part is very close to unity so it's worth rounding up to 6.
For the "reverse" calculation we rearrange the potential divider equation to R = (1000 * Vmeasured) / (Vreference - Vmeasured) = 6000 / (32854 - 6) = 0.18 ohm. That shows that you have an error in the "0.1 ohm" measurement, again obvious from the 16-bit value which should be 3, plus a fractional part if you really want to (and can) resolve into hundredths of an ohm.

In the above calculation, the subtraction can be done directly with 16 bit Words and the multiplication with PICaxe Basic's ** (for High Word) and * (for Low Word) operators. But first, Multiply the "1000" (R1) by 100 to give a result in hundredths of an ohm*. Put the High and Low words in the numerator of one of the "32-bit division" code snippets that I (or others) have posted and the result of the subtraction into the divisor and you should get a 16-bit result in hundredths of an ohm. My "simpler" double-word division routine only accepts a maximum divisor of 32767, so you might need to divide the numerator and divisor both by 2, or use my slightly more complex full 32-bit routine.

* EDIT: Sorry, make that 50 because 100 will overflow a word value. Then, for example, divide the divisor by 2 (which will also overcome the limitations of my "simple" division routine) to still give a result in hundredths of an Ohm (albeit with a resolution of 0.02 Ohm).

Cheers, Alan.
 
Last edited:

hippy

Technical Support
Staff member
It is a 24bit value we are working with.
Yes, but the ADC returns a 32-bit result. If we drop the top two bits and the bottom six it does become 24-bit. If we do that, what we should see from the ADC is ...

Vadc = Vref * R2 / ( R1 + R2 )

Nadc = $FFFFFF * Vadc / Vref

Nadc = $FFFFFF * R2 / ( R1 + R2 )

For R1 = 1K, R2 = 0.1, Nadc = 1677 ($00068D)

For R1 = 1K, R2 = 330, Nadc = 4162767 ($3F84CF)


Then to calculate R2 from the Nadc value ...

R2 = ( Nadc * R1 ) / ( $FFFFFF - Nadc )

For R1 = 1K, Nadc = 1677, R2 calculated = 0.0999669876459

For R1 = 1K, Nadc = 4162767 , R2 calculated = 329.99993341


But; if you aren't getting the correct Nadc value for what R2 is then the reversal won't be correct.
 

mortifyu

New Member
Hi All,

For those of you playing along at home...

The below readings were accurately taken with the original circuit modified for testing purposes with a 2.5V reference.

Code:
'This program takes 1000 samples from the LTC2485 and then provides the average values.

Startup:
#com 1
#terminal 4800
pause 3000

Init:
HI2cSetup I2CMASTER, $48, I2CFAST, I2CBYTE

main:
hi2cin (b1,b2,b3,b4)

w11=w11+1                'Total Samples Count
sertxd("Sample#",#w11,cr)    'Current Sample #
b5=b5+1
w3=w3+b1
w4=w4+b2
w5=w5+b3
w6=w6+b4

if b5=100 then
    w3=w3/100
    w4=w4/100
    w5=w5/100
    w6=w6/100
    b0=b0+1
    w7=w7+w3
    w8=w8+w4
    w9=w9+w5
    w10=w10+w6
    if b0=10 then gosub calc
    b5=0
    w3=0
    w4=0
    w5=0
    w6=0
endif

pause 150    'This brief delay required for LTC2485 to be ready to provide next conversion output.

goto main


calc:
w7=w7/10
w8=w8/10
w9=w9/10
w10=w10/10
sertxd("LTC2485 Average O/P after x1000 Samples: ",#w7," ",#w8," ",#w9," ",#w10,cr,lf)
b0=0
w7=0
w8=0
w9=0
w10=0
w11=0
return

CIRCUIT DIAGRAM
View attachment 0.1ohm Test Circuit.png

'VRef = 5.007VDC <<< MEASURED! This will improve when neatly on a professional PCB.
'ADC IN = 2.5000VDC <<< MEASURED!
'******************************************************
'32bits (4x Bytes) Returned by LTC2485
'%10111111 %11101100 %01001111 %10000000
'191 236 79 128
'*******************************************************
'24bit ADC Value (Extracted from 32bit read)
'%11111111 %10110001 %00111110
'255 177 62
'*******************************************************
'24bit Decimal Conversion
'16,757,054
********************************************************


This doesn't make ANY sense to me. I expected the ADC value to be about 8,388,607.

I triple checked my circuit and ultra fine (soldered under 60x mag) joints and everything is spot on. LTC2485 = 10-LEAD (3mm × 3mm) PLASTIC DFN

What am I missing I wonder?



LTC2485 <<< 24-bit ADC Datasheet
MAX6126AASA50+ <<< 5.000VDC Ref Datasheet
LM336-2.5V <<< 2.5000VDC Ref Datasheet




Kind regards,
Mort.
 

marks

Senior Member
Hi Mort,
I should be cheeky and just say thats seems spot on lol.
I did mention in post #54 In Common mode to get your 24bit result you need to divide by 2
so

Common Mode result 16,757,054 divide this by 2

result is 8378527 / 16777215 X 5.007 = 2.50049 v
probably all that noise that goes straight thru from those 7800's series regulators

I could probably try writing some code from your original results
I prefer just to work with 16 bit numbers but think I could extract 18bits and make it work
otherwise i think I'll leave it to others if you think you need the full 24 bit result
as i'm not ready to venture there yet.
 

AllyCat

Senior Member
Hi,

Yes, it looks correct to me as well, although I think the terminology may be a little confusing; I would prefer to consider the "top" bit (msb) of the raw data as the "overflow" indicator (when it is different to the second bit) and the second bit as the (two's complement) sign. You need to remember that this ADC has differential inputs and the "full-scale" (signed two's complement) is equal to the Reference voltage. From the data sheet:

"... a conversion result is generated for any differential input voltage VIN from –FS = –0.5 • VREF to +FS=0.5 • VREF. "

In your circuit, the negative (differential) input is grounded, but generally, it could be allowed to rise to VREF / 2 (and +FS remain close to zero/ground), which would generate a valid range of negative values. Therefore, you will need to ensure that the +Vin never exceeds Vref/ 2, but "errors" may allow +Vin to be slightly negative relative to the negative input (nominally earth), indicated by the second bit of the raw data being set.

Cheers, Alan.
 

hippy

Technical Support
Staff member
Yes, it looks correct to me as well, although I think the terminology may be a little confusing
I'd agree with that. Thanks to yourself and 'marks' for pointing that out.

With Vref = 5V then applying 2.5V in an ideal world would give a result of -
Code:
32-bit                             24-bit                     16-bit
%10111111111111111111111111111111  %111111111111111111111111  %1111111111111111
What you get isn't far from that value -
Code:
%10111111111011000100111110000000  %111111111011000100111110  %1111111110110001
Doing the reversal on those gives a calculated input of -
Code:
30-bit : 2.49699563021 V
24-bit : 2.49699577671 V
16-bit : 2.49702449073 V
If that 2.5V in were being created by an R1=1K / R2=1K divider, the reversal would calculate R2 to be -
Code:
30-bit : 997.599389094 R
24-bit : 997.599506018 R
16-bit : 997.622422386 R
So it seems to be about 0.15% accurate - guessed rather than calculated.
 

mortifyu

New Member
Marks, AllyCat & Hippy,

Thanks for that, yes, 'COMMON MODE' was previously mentioned and silly old Mort didn't trigger to it and take in Marks comments at POST #54.

In a way this shows the difference of 35 years of mostly self education/research/asking questions online VS a formal education. For some, it IS worth going to university while you're young enough to absorb it well and is the foundation as to why I personally sometimes get lost amongst the terminology in some cases.

Much as I generally understand MOST of the information in datasheets and can do and have done some pretty fancy things with electronics over the years, sometimes I read and suddenly a tune enters my head... Ground control to Major Tom... and I start thinking it's rocket science (even though it really isn't). :)



Regards,
Mort.
 

marks

Senior Member
Hi Mort,
cant you tell by the way I throw the code together I,m self taught too lol
its really a case of the data sheet being not quite right or helpfull
luckily with the testing you did it eventually got sorted out.
It will be interesting if you improve on those initial results ,I hope the test gear is uptoit!
yourve tempted me to try one of those adc but I think I'll start with something that seems
easier like the mcp4321.
Here's some code for you to tryout after penciling abit I thought I'd post it before its lost.
Rich (BB code):
#picaxe 08m2 
#terminal 4800
SYMBOL ADCvalue1 = W1   SYMBOL    X1 = W1   SYMBOL      R2 = W1 
SYMBOL ADCvalue2 = W2 
SYMBOL inX2 = W3 SYMBOL D2 = b7 SYMBOL D1 = b6 
SYMBOL Temp = W4 SYMBOL D4 = b9 SYMBOL D3 = b8
SYMBOL D5 = b11 SYMBOL rr = b11 SYMBOL R = b10
SYMBOL   index   = b12
Init:
HI2cSetup I2CMASTER, $48, I2CFAST, I2CBYTE
Main:
hi2cin (b5,b4,b3,b2)
CalibrationValues:
b5=%10000000 b4=%00000101 b3=%11111100 b2=%10111001 ' R2 @ 0.1 ohms (measured 0.12 ohms)
 ' b5=%10011111  b4=%11001001  b3=%10111010  b2=%10101100  ' R2 @ 330 ohms (measured 329.2 ohms) 
sertxd("LTC2485: ",#b5," ",#b4," ",#b3," ",#b2)
pause 200    'This brief delay is required between conversion requests from LTC2485.
Convert:    
 ADCvalue2 = ADCvalue2 *8 : ADCvalue2 = X1 **8 +ADCvalue2 ' extract equivalent 18bit value
 ADCvalue1 = $FFFF -ADCvalue2 +8 /10 +19660  ' calculate ADCvalue across R1(1000ohms)divide by 10

inX2 = $FFFF /X1 : Temp = $FFFF //X1+1
rr = X1 //10 : X1 = X1 /10
FOR index = 1 TO 4
inX2 = inX2 *10 : inX2 = Temp /X1 +inX2
r = inX2 //10 *rr
Temp = Temp //X1 *10 -r
      NEXT index
R2=ADCvalue2 ** inx2 
Uncalibrated: '  Max (333.33) ohmns
BinTOASCII R2 ,D5,D4,D3,D2,D1
sertxd(" R2 = ",D5,D4,D3,".",D2,D1) 'resolution 01 (000.17)(330.39)
calibrated: R2 = R2  Min 4 -4 **65309
BinTOASCII R2 ,D5,D4,D3,D2,D1
sertxd(" calibrated = ",D5,D4,D3,".",D2,D1,cr,lf) 'resolution 01 (000.12)(329.20)
end
 

mortifyu

New Member
Hi Mort,
cant you tell by the way I throw the code together I,m self taught too lol
its really a case of the data sheet being not quite right or helpfull
luckily with the testing you did it eventually got sorted out.
It will be interesting if you improve on those initial results ,I hope the test gear is uptoit!
yourve tempted me to try one of those adc but I think I'll start with something that seems
easier like the mcp4321.
Here's some code for you to tryout after penciling abit I thought I'd post it before its lost.

[/code]

Hi Marks,

Thanks for your input with this project and your fancy code.

BREAK OUT THE BEERS!!!! 🥳🥳🥳 Your code works an absolute treat!

It is so accurate, that it is now necessary for me to proceed to creating the test PCB jig in order to confirm the accuracy of this circuit/setup to the extent required.

At this stage I am pretty confident the accuracy will now reliably come down to test lead connections. This was the goal!




Kind regards,
Mort.
 

papaof2

Senior Member
A somewhat belated response to the original question of extended integer math would have been the Micromega uM-FPU which did 32 bit floating point and integer math and communicated with the PICAXE using I2C. The latest datasheet I can find is dated 2008 and while picaxestore.com lists the V2 and V3 uM-FPU both are "Out of stock".
I found one listing for it online: https://www.canakit.com/floating-point-co-processor-um-fpu-v3-1-com-08129.html
But it's anyone's guess whether any vendor still has them in stock.
 
Top