Forum - MCS Electronics

 

FAQFAQ SearchSearch RegisterRegister Log inLog in

VAL("0.001")=1.000E-3 (00)

 
Post new topic   Reply to topic    www.mcselec.com Forum Index -> BASCOM-AVR
View previous topic :: View next topic  
Author Message
Ev3658

Bascom Member



Joined: 30 Nov 2014
Posts: 22

russia.gif
PostPosted: Wed Dec 13, 2017 5:09 pm    Post subject: VAL("0.001")=1.000E-3 (00) Reply with quote

Faced the conversion problem in the operator VAL

Code:
Config Single = Scientific , Digits = 3
Dim S As String*250
Dim I As Single
S="0.001"
I=Val(s)
Print I           '=1.000E-3

S="0.002"
I=Val(s)
Print I           '=0.001

 

So is the problem of "Single" in "Word"

Code:
Dim I As Single
Dim X As Word
I = 0.003
X = I * 1000
Print X '=2


Is there a solution to the problem?

(BASCOM-AVR version : 2.0.8.0 , Latest : 2.0.7.8 )
Back to top
View user's profile
MWS

Bascom Member



Joined: 22 Aug 2009
Posts: 1813

blank.gif
PostPosted: Wed Dec 13, 2017 7:36 pm    Post subject: Re: VAL("0.001")=1.000E-3 (00) Reply with quote

Ev3658 wrote:
Code:
S="0.002"
I=Val(s)
Print I                                                     '=0.001

So is the problem of "Single" in "Word"
It's your problem, not Bascom's one.
With proper stack declaration the output of print is 0.002
Quote:
Code:
Dim I As Single
Dim X As Word
I = 0.003
X = I * 1000
Print X '=2

Is there a solution to the problem?
Try:
Code:
j = 0.003
J = J * 1000
X = J
Print X
The assignment of a single to a word - the single with limited resolution because of the variables layout with mantissa and exponent - will result in 2, because only the pre-decimal position of 2.99... will find use.
Going the approach as above results in some rounding with result 3.
Back to top
View user's profile
Ev3658

Bascom Member



Joined: 30 Nov 2014
Posts: 22

russia.gif
PostPosted: Thu Dec 14, 2017 10:24 am    Post subject: Reply with quote

Thank you!
Every day is a lesson Wink
Back to top
View user's profile
Display posts from previous:   
Post new topic   Reply to topic    www.mcselec.com Forum Index -> BASCOM-AVR All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum