View previous topic :: View next topic |
Author |
Message |
njepsen
Joined: 13 Aug 2007 Posts: 469
|
Posted: Fri Apr 01, 2022 4:46 am Post subject: Over run string size |
|
|
IMHO one of the most insidious errors in ones code is dealing a string of say 100 bytes, and somewhere in the code assigning >100 bytes to that string. Sometimes, depending on the order of the dim statements, the error might go un-noticed for months or even longer, because if the next variable in the dim list gets over written, it may not cause a problem; at other times, the results can be un predictable, so lately i have spending some time on a very LARGE price of code ,making sure that this cant happen. And thats not as easy as it sounds.
In the help for DECLARE FUNCTION, the help says:
Quote: |
When you want to pass a string, you pass it with it's name : string. So the size is not important. For example :
Declare function Test(s as string, byval z as string) as byte
You may however specify an optional maximum length. ] |
What does optional maximum length mean ? Does it mean if you exceed that length, only the declared length will be passed and no bad things will happen because of some inbuilt string length checking, or does it mean: DO NOT use more that the declared number of bytes?
DECLARE SUB
The declare sub help does not have the same clause with respect to the length of a string. Am i correct in thinking that the same comment applies?
Is this OK, as it seems to work OK?
Code: |
declare mysubname(somevariable as string) |
(BASCOM-AVR version : 2.0.8.5 ) _________________ Neil |
|
Back to top |
|
|
MWS
Joined: 22 Aug 2009 Posts: 2262
|
Posted: Fri Apr 01, 2022 9:31 am Post subject: Re: Over run string size |
|
|
njepsen wrote: | Code: | Declare function Test(s as string, byval z as string) as byte |
|
The keyword here is 'byval'.
First var 's' is handed over to the function by default method, which is 'byref' for strings. There it doesn't matter how big the string is, as only the string's address (pointer) is handed over.
However, if you declare a string 'z' as 'byval', a copy is created, IIRC within Frame space.
Depending on how much SRam you have already used up, you may want to limit the size of the copied string.
That's the help's message.
Quote: | Am i correct in thinking that the same comment applies?
Code: | declare mysubname(somevariable as string) |
|
Yes. 'somevariable" is handed over byref, costs little SRam and works on the original (referenced) string.
Quote: | Is this OK, as it seems to work OK? |
As always, it depends on your goal.
If you need to avoid that the original string is altered (which can be done by Sub AND Function), then use 'byval', if you need to alter the source-string, use 'byref' or without denominator. |
|
Back to top |
|
|
albertsm
Joined: 09 Apr 2004 Posts: 5913 Location: Holland
|
Posted: Fri Apr 01, 2022 10:01 am Post subject: |
|
|
A string does not carry length information. it only has an end marker.
this has pros and cons.
the optional length is the same for functions and subs. in fact i recently updated the help regarding this.
"When passing a string, it is recommended to also pass the maximum length the string can have : SomeString As String * 30 would indicate that the string will have a maximum length of 30 characters.
When you do not specify the length, and the compiler can not know the length because of nested sub calls, the compiler will reserve a space of the longest string from the framespace.
Please notice that you need to specify the string length in both the DECLARE and the actual implementation."
but this is only intended for allocating enough/the proper amount, of memory for temp strings like when passed with byval.
it does not mean there is some length check.
it is on my to do list but something else became more important. _________________ Mark |
|
Back to top |
|
|
njepsen
Joined: 13 Aug 2007 Posts: 469
|
Posted: Mon Apr 04, 2022 3:39 am Post subject: |
|
|
Thanks guys for that help. So just to be sure that Ive got this correct:
Byval & byref is straightforward.
If one does this:
Code: |
declare subname(somevar as string) |
without specifying the length of the string, the compiler reserves 254 bytes automatically. Mark you said
Quote: | the compiler will reserve a space of the longest string from the framespace |
so i assume 254 bytes?
But if you want to save space, and you know what the maximum string length is (say 50 bytes) then you would do this:
Code: | declare subname(somevar as string * 50)
|
and only 50 bytes get passed, and woe betide you if you try and store > 50 bytes. _________________ Neil |
|
Back to top |
|
|
albertsm
Joined: 09 Apr 2004 Posts: 5913 Location: Holland
|
Posted: Mon Apr 04, 2022 8:41 am Post subject: |
|
|
not completely correct.
the compiler tries to figure out the longest string used and will use that. it will not always use 254.
and when you specify a length, say 50, the compiler will use exact 50 bytes (+1 for the end marker). if you use it or not does not matter.
so simply said, it is a good idea to always specify the length of the string. _________________ Mark |
|
Back to top |
|
|
MWS
Joined: 22 Aug 2009 Posts: 2262
|
Posted: Mon Apr 04, 2022 11:50 am Post subject: |
|
|
njepsen wrote: | Byval & byref is straightforward. |
Really?
Quote: | If one does this:
Code: |
declare subname(somevar as string) |
|
If you do this, then it's automatically 'ByRef', thus a pointer. The Sub or Function does not have an influence on the length, as it was Dim'd elsewhere.
Only if you declare with 'ByVal', then a copy of the string within Frame is created.
Quote: | Quote: | the compiler will reserve a space of the longest string from the framespace |
so i assume 254 bytes? |
No, it will reserve 256 bytes, 255 bytes payload and the last byte as string-terminating zero.
Usage for ByVal in combination with strings is restricted anyway, as it costs SRam space and is slower due to the copy process.
It makes sense, if you want to alter a piece of string for any purpose while keeping the original string, also forwarding string-constants requires it. |
|
Back to top |
|
|
MWS
Joined: 22 Aug 2009 Posts: 2262
|
Posted: Mon Apr 04, 2022 1:01 pm Post subject: |
|
|
albertsm wrote: | and when you specify a length, say 50, the compiler will use exact 50 bytes (+1 for the end marker). |
Hmm, this is ambivalent.
If forwarded ByVal, the compiler reserves frame-space for 50+1 bytes, but the compiled code will copy the whole string even it exceeds specified length.
The compiler can not check dynamic behavior, only static one.
For dynamic length-checking, additional code needs to be included in run-time code.
Quote: | the compiler will use exact 50 bytes |
The way I understand your words is: If I use a string of 254 byte length as argument within such a sub:
Code: | Declare Sub test(ByVal S As String * 50) |
then the compiler takes care not to copy more than 50 bytes.
This is not the case and thus it is dangerous to wrongly assume this.
Even worse, if $framesize is set too small, the copied string may overwrite boundaries and destroys SW- and HW-Stack, finally crashes.
If tS in the example below has a length of 255, then exactly this happens.
Quote: | 'ver 2.0.8.5
$regfile = "m128def.dat"
$crystal = 8000000
$hwstack = 50
$swstack = 40
$framesize = 100
Dim tS As String * 52
Declare Sub test(ByVal S As String * 50)
tS = "01234567890123456789012345678901234567890123456789XX"
Call test(tS)
Call test("01234567890123456789012345678901234567890123456789XX")
Sub test(ByVal S As String * 50)
!NOP
End Sub |
|
|
Back to top |
|
|
albertsm
Joined: 09 Apr 2004 Posts: 5913 Location: Holland
|
Posted: Mon Apr 04, 2022 1:41 pm Post subject: |
|
|
the string does not carry length info. so only when you assign a constant and the length is known you can get an error or warning.
specification of the size is intended for this :
Code: | Function PrintZ() as String
Local s1 as String * 50
Local s2 as String * 50
s1 = "THIS IS TEXT IN VARIABLE S1, "
s2 = "this is text in variable s2"
logg s1 + s2
PrintZ = "finish"
End Function
Sub logg(Byval s as String)
Local planet as String * 16
planet = "Jupiter"
print s
End Sub
' main
Dim tmp as String * 50
tmp = PrintZ()
|
in this case the function adds 2 strings but the logg function does not know about the size. so the code will crash. when you specify that the passed param will be max 100 it will work.
so this size is only for that, it has nothing to do with checking lengths of assigned strings. _________________ Mark |
|
Back to top |
|
|
albertsm
Joined: 09 Apr 2004 Posts: 5913 Location: Holland
|
Posted: Mon Apr 04, 2022 1:50 pm Post subject: |
|
|
i will be bit more specific. the method used when a string is passed BYVAL so a copy is created:
- when a length is specified, it is used
- when a constant is used, the length of the constant is used. even if it is say 300 long.
- if the above does not apply, the longest dimmed string is used
but again, there is not check if a target string is overwritten. _________________ Mark |
|
Back to top |
|
|
MWS
Joined: 22 Aug 2009 Posts: 2262
|
Posted: Mon Apr 04, 2022 5:20 pm Post subject: |
|
|
albertsm wrote: | - when a length is specified, it is used |
How a length is specified?
I thought this would specify a length, as otherwise it would not make much sense:
Code: | Declare Sub test(ByVal S As String * 50) |
If I call this sub by:
Code: | Call test("01234567890123456789012345678901234567890123456789XX") |
then 52 chars plus terminating zero are copied to frame-space, easy to check in the simulator's memory window.
It does not look like any code, be it compile- or run-time, cares about the length.
Assigning a variable is not any different, while length information would be readily available.
Code: | Dim tS As String * 52
Declare Sub test(ByVal S As String * 50)
tS = "01234567890123456789012345678901234567890123456789XX"
Call test(tS) |
|
|
Back to top |
|
|
albertsm
Joined: 09 Apr 2004 Posts: 5913 Location: Holland
|
Posted: Mon Apr 04, 2022 7:36 pm Post subject: |
|
|
Quote: | How a length is specified? |
using the * length
Quote: | I thought this would specify a length, as otherwise it would not make much sense: |
sure i did not claim otherwise.
Quote: | then 52 chars plus terminating zero are copied to frame-space, easy to check in the simulator's memory window. |
yes it will be copied, but how much space if reserved from the frame space differs.
that is what i tried to explain.
using a constant it is indeed simple to test but it is not always so simple otherwise i would have implemented it already.
believe me or not. _________________ Mark |
|
Back to top |
|
|
MWS
Joined: 22 Aug 2009 Posts: 2262
|
Posted: Tue Apr 05, 2022 1:31 pm Post subject: |
|
|
albertsm wrote: | Quote: | How a length is specified? |
using the * length |
But the * length's only purpose is to calculate the frame-address for the next ByVal argument or Local variable within frame-space.
From your explanation I would expect some protection of the frame.
This becomes a "bomb" if used wrong, example:
Code: | $regfile = "m128def.dat"
$crystal = 8000000
$hwstack = 50
$swstack = 40
$framesize = 100
Declare Sub test(ByVal S As String * 160)
Call test("012")
Sub test(ByVal S As String * 160)
Local vs As String * 3
vs = "AAA"
End Sub |
"AAA" is placed behind the reserved space for S, overruns SW-/HW-stack borders and finally overwrites return address of the Sub test(), which crashes the controller.
Quote: | Quote: | I thought this would specify a length, as otherwise it would not make much sense: |
sure i did not claim otherwise. |
I meant a length with a more useful purpose.
Quote: | using a constant it is indeed simple to test |
I was more thinking about protecting the frame-space against overrun, which wouldn't be costly in terms of additional code, slower runtime maybe.
On the other hand, frame and stack can be destroyed by Local strings too.
Sure it is possible to leave it to the caution of the user, but then I would expect bigger exclamation marks in the help.
The help is IMHO not very clear in this point, otherwise njepsen would not have asked.
You know I am quite experienced, but what I have found out by testing, I wouldn't have expected at first glance.
Sure I believe you, but I try to see it from the user's point of view.
Strings, which are basically char arrays, are pretty dangerous in combination with Local or ByVal, and this deserves more attention.
The help for Local tells.
Quote: | Also local arrays are not possible. |
There may be a reason why.
PS, while I was testing:
Code: | Call test("012")
Sub test(ByVal S As String)
Local vs As String * 3
vs = "AAA"
S = "012345" '<--- This overwrites local vs
End Sub |
If length of the argument S is not specified, then location of local vs in frame-space is placed immediately behind the terminating zero of "012", writing a longer string to S overwrites the content of vs or other following locals.
Better avoid ByVal/Local in combination with strings or know exactly what you're doing. |
|
Back to top |
|
|
albertsm
Joined: 09 Apr 2004 Posts: 5913 Location: Holland
|
Posted: Tue Apr 05, 2022 3:03 pm Post subject: |
|
|
Quote: | This becomes a "bomb" if used wrong, example: |
sure but specifying too little space will always have that effect.
Quote: | From your explanation I would expect some protection of the frame |
that is why there is the $FRAMECHECK which will do that : add some code to see if too much room is claimed. but the normal code does not have this additional code.
but it is simple to add a check against overusage. i will add that.
Quote: | The help is IMHO not very clear in this point, otherwise njepsen would not have asked. |
no that can be more clear. but i think njepsen meant something different : when you assign a string to a string that is too small.
Quote: | Call test("012")
Sub test(ByVal S As String)
Local vs As String * 3
vs = "AAA"
S = "012345" '<--- This overwrites local vs
End Sub |
sure but why would one do that?
when you pass byval it makes not much sense to write to this variable. and those that want can specify the length.
but it is a good idea to show this in the help.
i do not get many support tickets about string problems but unlike the other data types they are indeed potential unsafe so i will reserve some time to see how it can be improved without rewriting all code. _________________ Mark |
|
Back to top |
|
|
MWS
Joined: 22 Aug 2009 Posts: 2262
|
Posted: Tue Apr 05, 2022 8:54 pm Post subject: |
|
|
albertsm wrote: | sure but specifying too little space will always have that effect. | I feel it tricky, because in this case the trigger is hidden.
Quote: | that is why there is the $FRAMECHECK which will do that | IMO $Framecheck is used if things already show to go wrong.
Quote: | but it is simple to add a check against overusage | Good, can be restricted to strings.
Quote: | but i think njepsen meant something different : when you assign a string to a string that is too small. | Exactly our topic:
njepsen wrote: | Does it mean if you exceed that length, only the declared length will be passed and no bad things will happen because of some inbuilt string length checking, | He asked for inbuilt string length checking if a string is bigger than the size the string argument's length allows.
This thread answered it: there is no such safeguard.
While njepsen's question did not specify ByRef or ByVal, I went for the ByVal argument, as only this one shows to be dangerous.
With ByRef:
Code: | Sub test(ByRef S As String * 50) | nothing will happen, still a pointer to the original string is handed over, the length argument is use- and harmless.
However, if the user expects clipping after 50 char, he may be disappointed too.
Quote: | Quote: | Call test("012")
Sub test(ByVal S As String)
Local vs As String * 3
vs = "AAA"
S = "012345" '<--- This overwrites local vs
End Sub |
sure but why would one do that? | Because if S already exists, why waste space and create another string to work with?
User's creativity to step into a booby-trap within 100 square mile of desert is unmatched.
Quote: | i do not get many support tickets about string problems | Maybe the fuss I made is the famous tempest in a teapot, but as njepsen was not sure about it, the fuss was maybe worth it.
Quote: | i will reserve some time to see how it can be improved without rewriting all code. | If it's ByVal, String and length, branch out to a special argument copy routine with length-check.
If length is missing, give a warning.
Make a ByVal string variable read-only or create a warning if write-accessed.
I also thought it would help, if Frame is growing downward, instead of upward. |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You cannot download files in this forum
|
|