View previous topic :: View next topic |
Author |
Message |
uhrmacher
Joined: 15 Sep 2005 Posts: 60 Location: Donauwoerth
|
Posted: Tue Aug 28, 2012 11:10 am Post subject: CF-Cards: AVR-DOS & IORDY?=no IORDY but RTS/CTS SOLVED! |
|
|
Hi AVR-Dos user,
at the Compact Flash cards, there is at Pin 42 of the CF-Card the IORDY I/O signal available. Purpose of IORDY is, to signalize to the controller, if the Card is
not ready to respond a data transfer request.
My question: Is it possible to configure AVR-DOS to use the IORDY? Background is, as far as i know, CF-Card applications, simply the Data written by AVR-DOS
to the Card, not paying attention if card is at this millisecond ready or not. With only a few data packages or slow data stream, it may not a problem (dataloggers etc..)
But at 230400baud, with a continuing full data stream it becomes essential to check the card behavior, means the card delays. AVR-DOS routine should
extend the transfer cycle when card is not ready. That causes in my application that always some bytes missing, which are not written to card, nevertheless the USART
has received correctly (GET) and the were written (PUT) but if card is busy at this milliseconds, data will go lost.
How AVR-DOS can be configured/enhanced to work with the IORDY or do have anyone another suggestion?
best regards and thank you for any recommendations!
Ingolf
Last edited by uhrmacher on Tue Jul 30, 2013 12:31 pm; edited 2 times in total |
|
Back to top |
|
|
albertsm
Joined: 09 Apr 2004 Posts: 5913 Location: Holland
|
Posted: Tue Sep 04, 2012 8:55 pm Post subject: |
|
|
i have asked the author of the CF card lib (and AVR-DOS) for his opinion. _________________ Mark |
|
Back to top |
|
|
uhrmacher
Joined: 15 Sep 2005 Posts: 60 Location: Donauwoerth
|
Posted: Wed Sep 05, 2012 8:07 am Post subject: |
|
|
Marc,
thank you for forwarding the request. I hope there will be good news.
Best regards.
Ingolf |
|
Back to top |
|
|
uhrmacher
Joined: 15 Sep 2005 Posts: 60 Location: Donauwoerth
|
Posted: Tue Oct 09, 2012 8:33 am Post subject: CF Card busy line |
|
|
Hi mark,
as a reminder of my request from 28.08.2012. Do you have meanwhile a opinion or statement received concerning a way to detect the CF-card busy while data writing?
best regards
Ingolf |
|
Back to top |
|
|
albertsm
Joined: 09 Apr 2004 Posts: 5913 Location: Holland
|
Posted: Tue Oct 09, 2012 10:05 pm Post subject: |
|
|
i have asked Josef but he is very busy. I asked him again. _________________ Mark |
|
Back to top |
|
|
uhrmacher
Joined: 15 Sep 2005 Posts: 60 Location: Donauwoerth
|
Posted: Wed Feb 06, 2013 11:28 am Post subject: still open topic |
|
|
Hello Marc,
i would ask if Josef was able meanwhile to think about the IORDY signal of the CF Card.
after tests with a 230400 baud generator who sends a ascii values of intermittent &H00 and &HFF,
i habe seen that if i configure on the cf-card recorder an arraysize of 4 bytes, it would work but with sporadic
errors in the recorded file. Arrays smaller or bigger causes much more errors, 5 to 30 errors within 4097 kbytes recorded.
Errors are always the same, a &H00 is missing or a &HFF is double.
The recorder must deal with the complete ASCII range from &H00 to &HFF.
as CF Card i use meanwhile the Hoodman 16GB which offers a theroretical speed of 120MB/s write and this type of
Card reduces a little bit the amount of sporadic errors comparing to formerly tests on some Sandisk 4GB (30MB/s)
As far as i have tested, it seems to be (that is what i believe..) that sometimes sporadic the card is still busy while "put" tries to write data to the card.
That is the point of question, if or not the put" routine prechecks before and while "putting" that the card is able to receive the data.
It may be the CF-Card IORDY signal.
not exactly shure if that is a bascom or avr related target, would you please so kind to discuss with Josef about this problem? I would be more than happy
if i could solve this problem which makes me really sleepless nights for so long time.
best regards
Ingolf
'-----------------------------------------------------------------------------------
codesnippet:
$regfile = "M2561def.dat"
$HwStack = 128
$SwStack = 128
$Framesize = 128
$crystal = 14745600
const serbuf = 254
const baudvalue = 230400
$baud= baudvalue
$baud1 = baudvalue
open "com1:" for binary as #1
open "com2:" for binary as #2
config serialin1 = buffered , size = serbuf
$include "CFG_CF_M128_.bas"
$include "CFG_AVRDOS.BAS"
const arraysize = 8
const flush_at = 2097152 // every 2MB do a flush...
dim i as word
dim r_dataarray(arraysize) as byte
dim w_dataarray(arraysize) as byte at r_dataarray overlay // with overlay, itworks with the smallest amount of errors at all
open "CFrectst.txt" for binary as #3
do
for i = 1 to arraysize
get #2 , r-dataarray(i)
next
put #3 , w_dataarray (1) ,, arraysize
bytecounter = bytecounter + arraysize
if bytecounter >= flush_at then
flush #3
end if
loop
'-------------------------------------------------------------------------------------------------- |
|
Back to top |
|
|
albertsm
Joined: 09 Apr 2004 Posts: 5913 Location: Holland
|
Posted: Thu Feb 07, 2013 9:49 pm Post subject: |
|
|
i have asked Josef again. Josef has a very busy day time job but usually he responds quick and has always helped by extending and fixing problems. I guess he missed this between the other questions.
Or his spam filter does not like that i send an url. _________________ Mark |
|
Back to top |
|
|
albertsm
Joined: 09 Apr 2004 Posts: 5913 Location: Holland
|
Posted: Fri Feb 08, 2013 10:07 pm Post subject: |
|
|
hi
Josef will look up his old driver where he used this pin.
He switched to read the status byte for 2 reasons :
- it is compatible with a hard disc which uses the same protocol but does not have this pin/signal
- pin busy is not much info. reading the status byte gives more info
In the mean time i had a look too.
I do not know if this IORDY signal must be 0 or 1 ?
But you could try it yourself :
- open the lib
- locate this code :
Code: | _Drive_Write_Sector1:
ld r22, z+ ; low byte
|
- then insert a wait which must be
*BASIC: BITWAIT PINX.Y, SET
where X.Y mist be your pin that is connected to the IORDY signal.
And this case shows that a valid (not busy) signal is '1', for '0' change SET to RESET
- then save it and try it.
you must enable the pullup of the used pin yourself. _________________ Mark |
|
Back to top |
|
|
oe9vfj
Joined: 17 Jun 2004 Posts: 269 Location: Austria, Hard
|
Posted: Tue Feb 12, 2013 11:30 am Post subject: |
|
|
Hi,
I think, checking the IORDY pin will not solve your problem.
The data-transfer to the card (Writing data to the card) is done independent of the way you give the bytes to AVR-DOS to save it. AVR-DOS collects 512 Bytes (1 Sector on the card) and then transfer this 512 Bytes at once to the card. This writing to the card is always at same speed. If your application is working correct with slow speed of incoming data, the writing part of data to the card is OK anyway.
The driver for the CF Card is checking the card for being ready to read or write data. This is done by the checking the status register of the card. The driver will not write data to the card, until the card is signaling to be ready. So current code has same effect as checking the IORDY pin.
The driver checks whether the card is ready to accept a new command and the card is ready to accept a data transfer. The timing of transferring the 512bytes of data to the card is done according to the CF-Card specification with 600 nSec cycle time. You see a delay of 500nSec (@genus(0.5) ) in the code of the driver and all other parts in the loop like putting the byte to the Data-pins and manipulating the strobe with set and reset the ATA_Port_DIOW-Pin, reducing and checking the byte-counter with cost more than 100nSec.
By the way, not all brands of CF-Cards support this pin 42 (some manuals says “not connected”, other “not asserted”).
I think your problem is on another side. When the FAT-Sectors need to be updated, it costs a lot of time. Earlier I made some tests and in worst case it gets up to appr. 200mSec. This is done, if AVR-DOS needs a free Cluster# from next FAT-Sector, because all cluster# of current FAT-Sector are consumed. Because of the way, the clusters (storage space on the card) of a files is chained in the FAT-Sectors, AVR-DOS has to load a new FAT-Sector to look for a free cluster, before this is can done, the old FAT Sector has to be saved to the card. The new cluster# need to be marked as EOF-Cluster and with this new Cluster# you have to load the old FAT-Sector again and store there the new Cluster#, so FAT has the info, where the space of a file will be continued. Than AVR-DOS has to save the old FAT-Sector again, because it was changed.
On a PC the OS can hold more than only one sector of FAT or Data-space in RAM. So most operation can done in RAM and portions of sector (4 or more) will be written to the card. On a ATMega with limited RAM, it is only possible to hold one sector of FAT and one sector of each file in RAM. CF-Cards and SD cards are optimized to read and write sectors in sequential order. The above mentioned process can not do this. So card will produce greater delay.
With your baud rate of 230400 you get a byte every 0.043 mSec and your buffer with 254 bytes will be filled up in 11,02 mSec. Writing a data sector will cost on a CF-Card depending of the vendor between 1 and 2 mSec. So this will be OK. But above mentioned process with FAT-Update will happen in best way on a not fragmented card with FAT16 and 2GB after appr. 16MByte, in all other situations earlier. During this Process you can get a greater delay then 11 mSec and here you get a buffer overrun and loose data.
I suggest to replace the serial buffer routine by an own routine, which can handle a greater buffer than 254 bytes.
I used this IORDY pin at the early beginning of my working with the CF-Card appr. 10 years ago. But after short time I changed to the current driver because of the facts, Mark mentioned in his post. Sorry to say, that I have no copy of this starting up driver. _________________ regards Josef
DOS - File System for BASCOM-AVR on http://members.aon.at/voegel |
|
Back to top |
|
|
albertsm
Joined: 09 Apr 2004 Posts: 5913 Location: Holland
|
Posted: Wed Feb 13, 2013 4:42 pm Post subject: |
|
|
The bigger buffer is one of the first possible solutions i have mentioned at support. I remember some example code too. I do not know how much memory you have for a buffer but i advise to increase it and test again. _________________ Mark |
|
Back to top |
|
|
uhrmacher
Joined: 15 Sep 2005 Posts: 60 Location: Donauwoerth
|
Posted: Tue Feb 19, 2013 8:57 am Post subject: |
|
|
Good morning Gentlemen,
as far as Josef explained, the bottleneck within the data writing process seems to be the FAT update and last sector to next sector chaining process to be. But this is a serious problem, because it seems to be that the processor is too long busy with the FAT update and meanwhile the USART input buffer becomes overflow.
In my opinion, any kind of software buffer will not help anyway because this procedures cannot take effect while the processor is busy with dealing the FAT and sector update and chaining.
I have studied intensively the ATMega 2561 data sheet about all around the USART but found no hint of the hardware buffer size, only the 2-level buffer is described. But in bascom, with config serialin the well known max 254 bytes are configurable.
Questions:
- is that 254 bytes buffer a configuration which uses a real hardware on the ATMega2561?
- are the 254 bytes the biggest possible size, limited by hardware or by Bascom?
I think about a independend from the program running serial input buffer like the existing but too small USART buffer.
Another way, to insert small portions of data reading relatively early into the FAT-update and sector chaining process, to avoid data overflow but this may be a little bit
wishful thinking.
Meanwhile i have clocked my prototype to 18.452MHz and i use also the fastest type of CF Cards from Hoodman ( 150MB/s read and 120MB/write ) but that has reduced the spurious data loss not significantely.
The suggestion with increasing buffer would work only if that buffer could act independent from what AVR-DOS routines doing while FAT is updating. As Josef explained, to buffer incoming data for the worst case of 200mS, it would be a 254/11*250 = buffer of 4618 bytes be necessary. But any kind of such a buffer will not run while processor is busy with avr-dos.. The ATMega2561 has total 256kbytes RAM with actual 10% used by bascom code.
Question: possible to create a AVR-DOS FAT update routine which checks at several times the USART buffer and gets only as much data bytes to safely prevent buffer overflow and write that data into an array variable which can be handled ( after FAT update has finished..) with the common PUT command? After that sequence, the normal get+put programm loop can can continue until the next necessary FAT update.
Maybe with a byte count to checkout at which time the next FAT update will come up.
I appreciate any kind of suggestions in that hard to work on target.
best regards
Ingolf |
|
Back to top |
|
|
albertsm
Joined: 09 Apr 2004 Posts: 5913 Location: Holland
|
Posted: Sat Mar 02, 2013 10:43 pm Post subject: |
|
|
Quote: | - is that 254 bytes buffer a configuration which uses a real hardware on the ATMega2561?
- are the 254 bytes the biggest possible size, limited by hardware or by Bascom? |
that buffer uses the SRAM. So yes that is real hardware.
the 254 is the biggest size. but in the support ticket you made you will find that i sent you some custom code to make a buffer of any size. it uses a word so can be much bigger.
the buffer only need to hold the data for the fat update, after that it will become empty again. _________________ Mark |
|
Back to top |
|
|
uhrmacher
Joined: 15 Sep 2005 Posts: 60 Location: Donauwoerth
|
Posted: Wed Mar 13, 2013 7:53 pm Post subject: 230.4kbaud recording on CF-Card |
|
|
Hello Marc,
yes i have evaluated the custom code, it runs but sporadic bytes will still fail. I have made the overall experience, that using of any kind of IRQ driven
functions or subs are contraproductive at this level of speed. I have written at least 10 different types of how to read/write, also by using parts in assembler
to read directly from UDR. Furthermore, in my case a word based buffer is not usable because i stronly need to read byte by byte, because of cocatenating
bytes while preprocessing results in special proprietary raw data of the javad. Using word variable can produce bad data.
I agree, this is a very specific application.
The best results, but never error free where catched with free running codes, without any USART IRQ.
As source by the way i have developed a 230400 baud rs232 pattern generator, it generates char patterns which easily can be checked out after recording.
At ne view you will see in a 4096 kB pattern if, for example space and Ü was send, if a byte was gone lost. In the past i have often used the real source but with
not repetitive serial data the troubleshooting is looking for the needle under the mountain
example:
Ü Ü Ü Ü Ü Ü Ü Ü Ü Ü Ü Ü Ü
Ü Ü Ü Ü Ü Ü Ü Ü Ü Ü Ü Ü Ü
Ü Ü ÜÜ Ü Ü Ü Ü Ü Ü Ü Ü Ü <- missing space error
Ü Ü Ü Ü Ü Ü Ü Ü Ü Ü Ü Ü Ü
Ü Ü Ü Ü Ü Ü Ü Ü Ü Ü Ü Ü Ü <- one char missing
in this manner you can find 5 to 13 errors in a 4096kB recording.
Is that possible to the source code of the AVR-DOS in my case to see how it can be modified to split maybe the delay into parts while reading and storing some data to avoid buffer overrun.
best regards
Ingolf |
|
Back to top |
|
|
albertsm
Joined: 09 Apr 2004 Posts: 5913 Location: Holland
|
Posted: Thu Mar 14, 2013 12:19 am Post subject: |
|
|
I do not understand why using a buffer would be a problem. and the remark about the word? we still use a byte buffer. but since we need a big array, we need words for the pointers.
i think i posted some code like this :
Code: | const cBufsize=5000
dim buf(cbufsize) as byte, rp as word,wp as word
do
if rp<>wp then
b=buf(rp) 'get from buffer
incr rp : if rp>cbufsize then rp=0
end if
loop
isr_rec_
buf(rp)=udr 'store data
inc rp
if rp>cbufsize then tp=0
return
|
the isr_rec could be done with nosave and some asm but if this delay already causes a problem, then i do not think it can ever work with FAT.
You could do a writesector where you write sector data without the FAT. but that requires a special app to read the disk data.
do you have an ISIS simulation? then i could have a look at it. _________________ Mark |
|
Back to top |
|
|
uhrmacher
Joined: 15 Sep 2005 Posts: 60 Location: Donauwoerth
|
Posted: Tue Jul 30, 2013 12:20 pm Post subject: solved problem :-) a report. |
|
|
Hi everybody related to this topic,
i struggled a very long time, how to record proprietary raw data, coming with 230400baud from a GNSS receiver ( GPS+Glonass+Beidou) to a CF Card.
There where always sporadic some bytes missing and that corrupts the data for post processing.
I tried a lot of different ways, to read and write as fast as possible and during my work it has become more and more painful clear that the chaining from
sector to sector takes "too much" time and while this sector chaining and FAT update work the USART buffer always becomes a overflow. I checked out that the time of
dealing with the FAT, could be from some milliseconds up to approx 200mS in worst case, depending on card size and occupied memory.
I tried the common SanDisk CF Cards with 30MB or 60MB /s and i tried also the Hoodman 16GB CF cards with 120MB/s write but that was not the master solution in this application, even because here is noting like UDMA
procedure used. So, card performance means not the root of the problem.
Any kind of interrupt solutions makes the problem more painful, so i left anything what has to to with interrupts.
I expanded the buffer by array, that lowers the amount of missing bytes but has not solved the problem and i tried dozen of different ways to read and write,
last but not least using also assembler code for the USART buffer and i wondered how fast Bascom works, the assembler codepiece within the bascom
does not makes a significant faster job than bascom code itself in that loop. So, the read/write code was as fast as possible but the sporadic missing bytes where still present....
More buffer was also no solution because of the bottleneck, that while the FAT update work blocks the controller, even the controller cannot read data to a buffer. If buffer is max. it means nothing because the controller is busy, not using them and meanwhile the USART input buffer becomes overflow. bascom defined buffer and/or array solutions at all are limited by this issue and the small USART buffer inside the ATMega is limited.
Then, one lonely day as i was under the shower, i checked out that i was always focused on my ATMega2561...i wondered why i have never tried to establish a RTS/CTS handshake.....
and - that was THE solution. I build up a using a second ADM202ARNZ for the RTS/CTS hardware circuit and reconfigured my bascom code and the receiver output.
Now, the data was recorded errorfree. WOW - cannot believe that...! I tried with the speed of 460800baud, the recorded data where still free from any simply missing byte.
Conclusion: recording raw serial binary with 460800baud to a CF Card, using AVR-DOS, no problem!
For anybody who is interested, please find a code snippet. It may be helpful for those who work on a
similar application.
best regards
Ingolf |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You cannot download files in this forum
|
|