calculation error!!!

[labcraft]

Has anyone tried using the modulus division command, but got errors? I have attempted to read a voltage, then transmit the voltage with the integer and decimal parts of the readings.
Since I am also using a voltage divider circuit, I factored that into the calculation.
This part of the code was as follows:

let w2=w1*60 ' multiplying by 60 gives an integer conversion ratio - 4096
let w3 =w2/4096
let w4=w2//4096
Running the simulator, the results were a returned figure of 14.2656 for w1=1000
Using windows calculator, the reading should have been 14.6484375

What's also bizarre about this is when I actually ran the program in the picaxe, the readings it returned were wildly fluctuating, anything from something near reality (11.85 volts) and voltage readings less than 1 volt, a multimeter confirming stable applied voltage.

Any clues as to what the #### is happening here?

 

[BeanieBots]

Can't say I've ever had a problem with it.
// gives the remainder.

If w0=1185
then w0/100=11
w0//100=85

 

[bgrabowski]

The 2656 figure is the remainder after dividing by 4096, not the decimal part of your answer. That is 2656/4096 which is 0.65 approx.

 

[hippy]

There's perhaps a misunderstanding as to how modulus works.

60000/4096 = 14 remainder 2656, it's that remainder you are perceiving as a decimal fraction which it isn't.

If you can give some further details on what you are reading and what that means, we can probably help with what the conversion should be.

As to the fluctuating readings, you'll have to give more information on how you are doing the reading and it may be necessary to examine the code being used.

 

[labcraft]

Thanx for help

I take your points! Remainder is NOT just the right hand side of the decimal point!
I will look into how else I can achieve my objective.

Thanks all for your help

Cheers,

Geoff

 

[hippy]

If you explain your objective in more detail; people here may be able to help and give advice.