Forum - MCS Electronics

Author Message
Ev3658

Joined: 30 Nov 2014
Posts: 22

Posted: Wed Dec 13, 2017 5:09 pm    Post subject: VAL("0.001")=1.000E-3 (00)

Faced the conversion problem in the operator VAL

 Code: Config Single = Scientific , Digits = 3 Dim S As String*250 Dim I As Single S="0.001" I=Val(s) Print I           '=1.000E-3 S="0.002" I=Val(s) Print I           '=0.001

So is the problem of "Single" in "Word"

 Code: Dim I As Single Dim X As Word I = 0.003 X = I * 1000 Print X '=2

Is there a solution to the problem?

(BASCOM-AVR version : 2.0.8.0 , Latest : 2.0.7.8 )
MWS

Joined: 22 Aug 2009
Posts: 1813

Posted: Wed Dec 13, 2017 7:36 pm    Post subject: Re: VAL("0.001")=1.000E-3 (00)

Ev3658 wrote:
 Code: S="0.002" I=Val(s) Print I                                                     '=0.001

So is the problem of "Single" in "Word"
It's your problem, not Bascom's one.
With proper stack declaration the output of print is 0.002
Quote:
 Code: Dim I As Single Dim X As Word I = 0.003 X = I * 1000 Print X '=2

Is there a solution to the problem?
Try:
 Code: j = 0.003 J = J * 1000 X = J Print X
The assignment of a single to a word - the single with limited resolution because of the variables layout with mantissa and exponent - will result in 2, because only the pre-decimal position of 2.99... will find use.
Going the approach as above results in some rounding with result 3.
Ev3658

Joined: 30 Nov 2014
Posts: 22

 Posted: Thu Dec 14, 2017 10:24 am    Post subject: Thank you! Every day is a lesson
 Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First
 All times are GMT + 1 Hour Page 1 of 1

 Jump to: Select a forum BASCOM AVR/8051----------------BASCOM-AVRBASCOM-8051BASCOM-ARDUINOShare your working BASCOM-8051 code hereShare your working BASCOM-AVR code hereBASCOM BETA-SLA BASCOM Related----------------EASY TCP/IPAVR-DOSAR7212KokkeKat FAT-free SD card libBASCOM Project Blog Other Stuff----------------VariousPCB'sRoboticsNew WebSiteAnnouncementsAVR Archive----------------BASCOM-AVR ArchiveBASCOM-8051 ArchiveBASCOM-AVR Unsupported versionsEasy TCP/IP ArchiveBASCOM-EDB
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum