Posted: Wed Dec 13, 2017 7:36 pm Post subject: Re: VAL("0.001")=1.000E-3 (00)
Ev3658 wrote:
Code:
S="0.002"
I=Val(s) Print I '=0.001
So is the problem of "Single" in "Word"
It's your problem, not Bascom's one.
With proper stack declaration the output of print is 0.002
Quote:
Code:
Dim I AsSingle Dim X AsWord
I =0.003
X = I *1000 Print X '=2
Is there a solution to the problem?
Try:
Code:
j =0.003
J = J *1000
X = J
Print X
The assignment of a single to a word - the single with limited resolution because of the variables layout with mantissa and exponent - will result in 2, because only the pre-decimal position of 2.99... will find use.
Going the approach as above results in some rounding with result 3.
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You cannot download files in this forum