Problem with TCPREAD function

bgardner · Joined: 09 Apr 2004 Posts: 46 Location: Everett, WA

Greetings,

I have a simple web server running in an atmega328P based on the sample web server code from the sample files. I'm using a WIZ810MJ module with the W5100 chip and using the latest Bascom 2.0.7.6. I'm attempting to read POSTed data from the browser. Like the sample, I'm using a DO LOOP to read the header information until I get to a blank line which signifies the end of the header info. According to the help file:

"When you use TCPread with a string variable, the routine will wait for CR + LF and it will return the data without the CR + LF.
For strings, the function will not overwrite the string."

"For example, your string is 10 bytes long and the line you receive is 80 bytes long, you will receive only the first 10 bytes after CR + LF is encountered.
Also, for string variables, you do not need to specify the number of bytes to read since the routine will wait for CR + LF."

My problem is that TCPREAD is not waiting for a CRLF until it returns data. Sometimes it returns a partial string before the CRLF. This does not cause a problem until the string returned is just before the CRLF. When this happens, the next TCPREAD sees just the CRLF and thinks that this is the end of the header. Here's a dump to a serial terminal demonstrating the problem:

Listening on socket : 1
sock_est
receive buffer size : 537
S=POST /index.html HTTP/1.1
S=Host: 192.168.0.137:5001
S=Connection: keep-alive
S= 7
S=Cache-Control: max-age=0
S=Accept: text/html,application/xhtml+xml,appl
S=ication/xml;q=0.9,image/webp,*/*;q=0.8
S=Origin: http://192.168.0.137:5001
S=User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW
S=64) AppleWebKit/537.36 (KHTML, like Gecko) C
S=hrome/32.0.1700.76 Safari/537.36
S=Content-Type: application/x-www-form-urlenco
S=ded
S=Referer: http://192.168.0.137:5001/index.htm
S=l
S=Accept-Encoding: gzip,deflate,sdch
S=Accept-Language: en-US,en;q=0.8
S=
Posted data
IP=1234

You can see that several of the header entries are split without waiting for the CRLF. The specific area where I encountered the problem is the 'Referer:" header. If I change the IP address to 192.168.0.37, then the TCPREAD ends just before the CRLF and the next TCPREAD sees the CRLF as a blank line and the end of the header. In that case my POSTed value ends up being "Accept-" which are the next 7 characters. Since this post is getting a little long and the code is in the sample file, I have left the code out of the post but, if necessary, I can certainly provide the relevant part of it.

I'm not sure if this is something I'm doing wrong, a bug, or my misunderstanding. Any help appreciated.

Thanks,

Bob

albertsm · Joined: 09 Apr 2004 Posts: 5916 Location: Holland

while i implemented this function, waiting for a cr+lf might not be a good idea in case such data will not arrive.
A better way is to check if there is data pending in the buffer and to read it and process it using a state machine.
Here is a code fragment of what i mean :

bgardner · Joined: 09 Apr 2004 Posts: 46 Location: Everett, WA

Thanks Mark. I'll give this a try and see how it goes.

Bob

bgardner · Joined: 09 Apr 2004 Posts: 46 Location: Everett, WA

Hi Mark,

So I'm working on implementing your suggestion but it is more complicated and I'm trying to figure out why I need to do it this way. When I implement the Socketstat command, the result is the number of bytes in the complete header so it's all in the buffer but then when I use the Tcpread command in a loop to extract data from the buffer, I either get a complete line up to the CRLF or a partial string of exactly 46 bytes. This is very consistent and does not seem to match what is in the help file. It seems like if all of the data is in the buffer, the Tcpread should read up to the CRLF instead of stopping at 46 bytes if the string is longer. So do I not understand this correctly?

I'm not complaining - just trying to understand why this isn't working correctly. I have been a very pleased Bacom user for a long time and am very satisfied.

Thanks,

Bob

albertsm · Joined: 09 Apr 2004 Posts: 5916 Location: Holland

Bob

i just show an alternative. using the tcpread will work too. but you must make the string long enough to hold the line. if you make it too small, you will get fragments in order not to overwrite the string. but these fragments are hardly useful when you want to process the data.
tcpread () with a string works like this :
- it get 1 byte from the buffer
- when it is not 13 or 10 it stores it in the buffer when it fits. otherwise it will return the received data.
- when 13 is found, it is not stored. it is just skipped.
- when 10 is found, it marks the end of the line and the routine ends.

i suspect your dimmed string is 46 bytes. otherwise attached the exact used program.

bgardner · Joined: 09 Apr 2004 Posts: 46 Location: Everett, WA

Mark,

This latest information helped me solve the problem. I was basing my program on the web server example from the samples and that sample used the same string variable several places. One of the places was to hold the HTML page and, since mine was longer than 250 bytes, I had dimensioned it to 300 bytes and used the $Bigstrings option. That worked great for the web page, unfortunately that same string variable was what I was using in the TCPread function and the use of this big string confused the TCPread function. I assumed this was the problem and created another variable for use in the TCPread function that was dimensioned less than 255 and the program now works fine.

Thanks again for all your help. You have always provided great support.

Bob