Forum - MCS Electronics

 

FAQFAQ SearchSearch RegisterRegister Log inLog in

Errors with main.bas program
Goto page Previous  1, 2, 3
 
Post new topic   Reply to topic    www.mcselec.com Forum Index -> KokkeKat FAT-free SD card lib
View previous topic :: View next topic  
Author Message
KokkeKat

Bascom Member



Joined: 08 May 2011
Posts: 59
Location: Stockholm

sweden.gif
PostPosted: Fri Jul 08, 2011 6:30 pm    Post subject: Reply with quote

Hi Neil

There is currently no support for deletion of file or subdirectory. While I can see the benefit from having it, I haven't found the time to develop it. If you would like to contribute, please share your code.

What you need to do is:
A restore each "record" in the entire FAT chain to 0 (so it becomes available)
B set the first byte of the (sub)directory entry to &HE5 (so it gets marked as "reusable")
C (not absolutely necessary: write 0 to every byte in the clusters that contained the file or subdirectory data)
D update the fsinfo variables to reflect the "regained" cluster space
E anything I just failed to mention

I would recommend not keeping (reusing) the directory entry, but that's your decision.

---

The file writing loop you mention could be a bug in the library. The number 16 is likely to come from the fact that each directory sector contains a maximum of 16 directory entries. (16 * 32 = 512.) This could be that the transition from the first to the second directory cluster isn't working (in case of 1-sector-per-cluster) or that the transition from the first to the second directory sector isn't working (in case of >1-sector-per-cluster). Could you please post your code so I can debug it?

It sounds like you need to format the SD card. I also recommend using a hex editor. I bought Winhex, but I'm sure there are others. Such a software would enable you to see (and change) the actual data stored on your SD card.

Kind regards

Niclas
Back to top
View user's profile
njepsen

Bascom Member



Joined: 13 Aug 2007
Posts: 418

newzealand.gif
PostPosted: Sat Jul 09, 2011 4:08 am    Post subject: add file to directory Reply with quote

Niclas,
I asked how a file is referenced back to its parent directory. Your answer "Once you have read up on FAT32, you'll know the answer to this question." is not helpful and is condescending - I have read your and Paul's and many other docs on the subject, but its not clear how this part works. I think it goes something like this: //A 32 byte directory entry resides in a sector of 16 such entries. Each entry( directory record) has an 8.3 name, atribute ( indicating file or dir, hidden etc ), cluster address of the FIRST cluster of the file and file size.( but what about when there is no file in the directory yet - how can it have a first cluster and size?) If the file occupies more than one cluster, which I think is 32k, then the NEXT cluster addr used by that file is given in the FAT record for the previously used cluster. I.e each FAT entry points at the next available cluster, even if it hasnt been used yet ( I'm not sure about this bit?). //
This still doesnt help me when I want to add a file to a subdir. Obviously I need to locate the cluster addr recorded in the directory entry, NOT the cluster that the directory entry resides at. Do I do the file write using SDstartdircluster with this addr or do I have to read the next FAT addr ( which will be the same if the file size is <32k?)

In either case Where or how i find or calculate the appropriate variable. What is the appropriate variable?

Guess I'll pass on the deleting.
I'll post the code which writes a max of 16 files shortly. Also, If I create directories in a loop, incrementing the dir name each time round the loop, it also only creates 16 directories, then overwrites the last one. Same fault?

regards
neil

_________________
Neil


Last edited by njepsen on Sun Jul 10, 2011 11:25 am; edited 1 time in total
Back to top
View user's profile
njepsen

Bascom Member



Joined: 13 Aug 2007
Posts: 418

newzealand.gif
PostPosted: Sat Jul 09, 2011 4:47 am    Post subject: Maximum file number = 16? Reply with quote

Niclas,
here is the code that I used to continuously loop making files or directories. In either case, only 16 were created ( according to my PC)

neil

$regfile = "m328pdef.dat"
$crystal = 8000000
$framesize = 40 ' This lib only uses ~10 $hwstack - please see the application note for more information
$hwstack = 40 ' This lib only uses ~10 $hwstack - please see the application note for more information
$swstack = 40 ' This lib only uses ~10 $hwstack - please see the application note for more information
$include "KokkeKat_FAT-free_SD_lib_declarations.bas"

' '-------------------------------------------------------------------------------
'Open a TRANSMIT channel for debug
Open "comd.4:9600,8,n,1" For Output As #1
Dim Sddebug As Byte ' Only needed for the examples
Sddebug = 0 ' Only needed for the examples

Dim Filename As Integer
Dim Filenamestr As String * 10

Filename = 200
Dim S As Integer

Do

Filename = Filename + 1
Filenamestr = Str(filename)
Sdentrynames = "000 TXT"
Mid(sdentrynames , 1 , 3) = Filenamestr
Sdstatus = 0
Gosub Sdinit
Print #1 , "init done"
If Sdstatus = 0 Then
Gosub Sdinitfs
Wait 1
' Gosub Make_file
Gosub Make_directory
Loop
Else
Print #1 , "bad sdstatus after SDinitfs"
End If
End ' End program

$include "KokkeKat_FAT-free_SD_lib_code.bas"
'===============================================================================
Make_directory:
Print #1 , "New dir name is " ; Sdentrynames
Sdcreatemode = 1 ' 1=Create subdirectory
Sdyear = 2011
Sdmonth = 2
Sdday = 17
Sdhours = 00
Sdminutes = 30
Sdseconds = 32
Gosub Sdreadfsinfo
Sdstartdirclusterd = 0 ' Root dir
Gosub Sdcreatefileordir ' Find free FAT entry, find free directory entry, and save the directory entry back to the SD card
Gosub Sdwritefsinfo 'neil added
Return
'========================================================================

Make_file:
' Create and write to the file
Print #1 , "filename =" ; Sdentrynames
Sdcreatemode = 0 ' Create file
Sdstartdirclusterd = 0 'change this if writing to subdirectory?
'Sdstartdirclusterd = sdfatclusterd
Print #1 , "sdfatclusterd=" ; Sdfatclusterd
Print #1 , "sdclsterd = " ; Sdclusterd
Sdcreatemode = 0 ' i.e a file
Gosub Sdcreatefileordir ' Find free FAT entry, find free directory entry, and save the directory entry back to the SD card
Sdstatus = 0
For Sdtempw2 = 1 To 512
Sdbuffer(sdtempw2) = 0
Next Sdtempw2
Sdtempb2 = 65 ' Write "A"
Sdtempw2 = 0
While Sdclosefile = 0 And Sdtempw2 < 1024 ' 512 times (always make sure that Sdclosefile = 0 before writing)
Incr Sdtempw2
Sdbyterw = Sdtempb2
Gosub Sdwritebyte ' Save a byte to the card
Wend
Sdtempb2 = 66 ' Write "B"
Sdtempw2 = 0
While Sdclosefile = 0 And Sdtempw2 < 1024 ' 512 times (always make sure that Sdclosefile = 0 before writing)
Incr Sdtempw2
Sdbyterw = Sdtempb2
Gosub Sdwritebyte ' Save a byte to the card
Wend
Sdtempb2 = 67 ' Write "C"
Sdtempw2 = 0
While Sdclosefile = 0 And Sdtempw2 < 1024 ' 512 times (always make sure that Sdclosefile = 0 before writing)
Incr Sdtempw2
Sdbyterw = Sdtempb2
Gosub Sdwritebyte ' Save a byte to the card
Wend
Sdtempb2 = 68 ' Write "D"
Sdtempw2 = 0
While Sdclosefile = 0 And Sdtempw2 < 1024 ' 512 times (always make sure that Sdclosefile = 0 before writing)
Incr Sdtempw2
Sdbyterw = Sdtempb2
Gosub Sdwritebyte ' Save a byte to the card
Wend
Sdbyterw = 69
Sdclosefile = 0
Gosub Sdwritebyte
Gosub Sdfinalizeafterwriting ' After writing the last byte, close the file
Gosub Sdwritefsinfo ' Save the updated fsinfo variables back
Return
'=======================================================================

_________________
Neil
Back to top
View user's profile
KokkeKat

Bascom Member



Joined: 08 May 2011
Posts: 59
Location: Stockholm

sweden.gif
PostPosted: Sun Jul 10, 2011 9:12 pm    Post subject: Reply with quote

Dear Neil

I have chosen to share some of my development with the community as a way to "pay forward" the help I have received from other people. Some of my development can be found as projects on www.avrfreaks.net and this SD library seemed fit for a proper publication as an application note at www.mcselec.com. As I want this library to be actively used and improved by contributions from other users (and I also want to get help in identifying bugs), I asked MCS Electronics to provide this support forum. Please notice that I am not an employee of MCS Electronics and that there is no commercial relationship between me and MCS Electronics.

Please understand that I do this out of my own spare time and that this support forum is the place where users can report real or suspected bugs in the library code. This is not a support forum for the FAT32 file system and it is not a FAT32 beginner's help-me-along forum. The user must be prepared to learn a lot by himself, including reading the library code for clues about how it works.

This is the reason why I have pointed out what you need to learn with regards to FAT32, but (having already spent a considerable amount of time helping you) I have not answered out-of-scope questions.

Nevertheless, in response to your summary: You are getting close: The root directory always has a volume ID. A subdirectory always has one "." entry pointing to itself and one ".." entry pointing to the directory above. I am not sure, but I think that a directory entry for a file can (according to FAT32 specs) have a null pointer, before there is any file content. My lib always allocates a cluster when the directory entry is created, so it will always point to a cluster. A directory never has a size - only files have a size. You are also almost right about the FAT chain. Each FAT entry points to the next USED cluster. The last (used) FAT entry in the chain has an EOC (End-Of-Chain) marker >=&HFFF8 or >=&H0FFFFFF8. A file or subdirectory using <= 1 cluster has an EOC marker in its (only) FAT entry.

When you want to create a file in a subdirectory, you must know beforehand which is the first cluster of this directory's cluster chain. This you find by starting at the root directory and looking up the particular subdirectory's first cluster (comparing 8.3 names). This is done iteratively (in case of multiple directory levels) until you know the first cluster of the subdirectory in which you want to create the file. If you use Sddirlist, it will be found in Sddirlistdirentry(). This value you then assign to Sdstartdirclusterd before calling Sdcreatefileordir to create the new file. Please see the code sample I posted in thread "List SUBDir". It is fairly close to what you need.

At the time of (=just before) creating a file, you don't know in which cluster it will start. As you can see in the library code for Sdcreatefileordir, it first looks up an available directory entry (extending the directory if necessary). Then it identifies the next available FAT entry (which corresponds to what becomes the first cluster of this file). You won't know the file's or subdirectory's first cluster number until the return from Sdcreatefileordir.

Thanks for posting your code. I will run and debug it in the coming week. There could be a library bug in the transition from the first (sub)directory sector to the next. In that case I will try to reply with an updated version before Saturday (but no guarantees).

Please notice that you should only initialize the SD card when you "boot up" the AVR. So, place Gosub Sdinit and Gosub Sdinitfs before the main program loop. Also, make sure that Sdentrynames is always 11 characters long. It could be that the forum text editor omits consecutive space characters (at least the leading indentations are cut out), but just to be sure: Use Sdentrynames = "000_____TXT". (If I remember correctly, the use of strings is a bit "dangerous". If you first do Mystring = "ABCDEFGH" and then Mystring = "IJK", I think the RAM location storing Mystring will actually contain "IJK[ASCII number 0]EFGH[ASCII number 0]". I am overlaying Sdentrynameb() on top of Sdentrynames, and the lib always "really" uses Sdentrynameb(). The string variable is "terminated" by a 0 value (which is why a string declaration must be one byte longer than the actual string). (Several consecutive space characters (ASCII 32) should also work, but I'm afraid it won't "print well" in the forum.)

Kind regards

Niclas
Back to top
View user's profile
njepsen

Bascom Member



Joined: 13 Aug 2007
Posts: 418

newzealand.gif
PostPosted: Mon Jul 11, 2011 3:35 am    Post subject: Reply with quote

Niclas,
Your reply //"If you use Sddirlist, it will be found in Sddirlistdirentry(). This value you then assign to Sdstartdirclusterd before calling Sdcreatefileordir to create the new file"?? was exactly the pointer I needed. i can get the lo byte & hi byte from the array, and assign as reqd.

I know where you are coming from, and I know that you are doing this is your spare time, and I DO appreciate the help you have given me, and also realiase the enoromous effort you must have put into the project so far. However from my point of view, when i looked at the initial description of the library, I rightly or wrongly assumed that I could simply use it as a tool and drop it into my other code. This is but a small part of a large commercial effort that i am involved in and I simply dont have the time or the will to learn about the complexities of FAT32 and reading writing to SD cards -- having said that you have forced me ( which i am now grateful for) yelling and screaming down that path. However until I can read and write to directories, delete files from a directory, and handle read/write errors, i cannot progress with the rest of the project, which is quite large. To this end, if you are willing, I would like to put this on a commercail footing so you are not helping me for nothing. I would like to discuss this further with you- my email is [removed by KokkeKat for spam reasons],. or alternatievly we could talk via skype. i will respect your privacy if you say no. Thankyou for looking at the 16 file thing - thats not important right now, so there is no hurry. I was aware that I should only do SDinit once, but was too lazy to move it right now.

regards
neil

_________________
Neil
Back to top
View user's profile
KokkeKat

Bascom Member



Joined: 08 May 2011
Posts: 59
Location: Stockholm

sweden.gif
PostPosted: Mon Jul 11, 2011 10:21 am    Post subject: Reply with quote

Hi Neil

Thank you for understanding. I will contact you on your private e-mail and we can continue the commercial conversation in private. I have edited your posting (removing your e-mail address) for spam reasons. If you want your e-mail address to be visible to everybody, please change it back.

I am now starting to look at the 16 file example, in case there is a library bug.

Kind regards

Niclas
Back to top
View user's profile
njepsen

Bascom Member



Joined: 13 Aug 2007
Posts: 418

newzealand.gif
PostPosted: Mon Jul 11, 2011 11:06 am    Post subject: Reply with quote

Thanks. I was intending to do the same.
_________________
Neil
Back to top
View user's profile
KokkeKat

Bascom Member



Joined: 08 May 2011
Posts: 59
Location: Stockholm

sweden.gif
PostPosted: Mon Jul 11, 2011 12:30 pm    Post subject: Reply with quote

Hi Neil

There is indeed a library bug in Sdcreatefileordir:. Thanks for helping me find it. Please add the two lines (at about row 1400) I have highlighted below:

Code:

      ' An available (previously erased) directory entry or EOD End Of Directory marker found
      ' Sddirclusterd, Sddirsecincluster, and Sddirbufferpos now point to this directory entry
      If Sdstatus = 28 Or Sdstatus = 32 Then                ' Free directory entry found
         Sdstatus = 0                                       ' Remap Sdstatus

         ' Find free FAT1 entry and store position in Sdfatsectornum, and Sdfatbufferpos plus Sdfatclusterd
         Sdsectord = Sdfatsectornum                         ' Continue looking in the same FAT sector
         Sdbufferpos = Sdfatbufferpos
         Gosub Sdfindfreefat
         If Sdstatus = 0 Then

            Gosub Sdlocatedirsector                         ' Point to the available directory entry again
            Sdsectord = Sdsectord + Sddirsecincluster       '<--- new line
            Decr Sdsectord                                  '<--- new line

            #if Sdwmode = 1
               Sdreaddestination = 0
            #endif
            Gosub Sdreadsector
 


I have made some minor adjustments to your code (regarding filename string handling). This works when creating 130 subdirectories on my 4 GB (8 sectors per cluster) SDHC card: (In the root directory of an 8-sectors-per-cluster card, there will be the volume ID and room for 127 file or subdirectory entries. When creating 130 subdirectories, it extends the root directory cluster, which is the next "critical level" to test.)

Code:

$regfile = "m644def.dat"
$crystal = 8000000
$framesize = 40                                             ' This lib only uses ~10 $hwstack - please see the application note for more information
$hwstack = 40                                               ' This lib only uses ~10 $hwstack - please see the application note for more information
$swstack = 40                                               ' This lib only uses ~10 $hwstack - please see the application note for more information
$include "KokkeKat_FAT-free_SD_lib_decl.bas"
$include "3310decl.bas"                                     ' <----- This is a display library I used when developing, so it's not required by the SD card lib

' '-------------------------------------------------------------------------------
'Open a TRANSMIT channel for debug
'Open "comd.4:9600,8,n,1" For Output As #1
Dim Sddebug As Byte                                         ' Only needed for the examples
Sddebug = 0                                                 ' Only needed for the examples
Dim Sdtempw2 As Word                                        ' Only needed for the examples

Dim Filename As Integer
Dim Filenamestr As String * 11

 ' Display initialization
Call D3310reset                                             ' <----- This is a display library I used when developing, so it's not required by the SD card lib
Call D3310init                                              ' <----- This is a display library I used when developing, so it's not required by the SD card lib
Call D3310clear                                             ' <----- This is a display library I used when developing, so it's not required by the SD card lib


Sdstatus = 0
Gosub Sdinit
If Sdstatus = 0 Then
   Gosub Sdinitfs
   If Sdstatus = 0 Then
      Filename = 100
      Do
         Incr Filename
         Filenamestr = Str(filename)
         Sdentrynames = "000_____TXT"
         Mid(sdentrynames , 1 , 3) = Filenamestr
         ' Gosub Make_file
         Gosub Make_directory
         Wait 1
      Loop Until Filename = 230
   End If
End If

End

$include "KokkeKat_FAT-free_SD_lib_code.bas"
$include "3310endshiftout.bas"                              ' <----- This is a display library I used when developing, so it's not required by the SD card lib

'===============================================================================

Make_directory:
   ' Print #1 , "New dir name is " ; Sdentrynames

   Call D3310position(0 , 0)
   Call D3310print(sdentrynames)

   Sdcreatemode = 1                                         ' 1=Create subdirectory
   Sdyear = 2011
   Sdmonth = 2
   Sdday = 17
   Sdhours = 00
   Sdminutes = 30
   Sdseconds = 32
   Gosub Sdreadfsinfo
   Sdstartdirclusterd = 0                                   ' Root dir
   Gosub Sdcreatefileordir                                  ' Find free FAT entry, find free directory entry, and save the directory entry back to the SD card
   Gosub Sdwritefsinfo                                      ' neil added
Return
 


Kind regards

Niclas
Back to top
View user's profile
KokkeKat

Bascom Member



Joined: 08 May 2011
Posts: 59
Location: Stockholm

sweden.gif
PostPosted: Mon Jul 11, 2011 1:00 pm    Post subject: Reply with quote

Hi Neil

Just a quick notice concerning Windows file system buffering. In the example above, when creating so many subdirectories that it extends the directory cluster chain, it didn't seem to help that I ejected the SD card in Windows before putting it in the AVR socket. I still had to reboot my computer for Windows to care about the entries in the extended cluster (subdirectories 228-230).

Also, another thing is that unless you are going to use a fixed SD card size (the sectors per cluster concern again), your application should adjust the amount of data it writes to each file, so that it always makes good use of whatever card size you insert. Example: if you decide on a fixed file size of 4 kB (because you use a 4 GB card) and then insert an 8 GB card, it will still only be able to store roughly the same amount of data (because you waste the additional sectors of the (only) cluster each file requires. (You will still benefit to some degree from the additional space with regards to the directories' cluster usage.)

Finally, my lib should be well equipped for handling read and write errors. In case of one occuring, you might need to do the Sdinit and Sdinitfs again, but in that case, you need to make sure that you first set all relevant "system" variables to zero, so as to avoid some of them adding up each time you re-initialize.

Kind regards

Niclas
Back to top
View user's profile
Display posts from previous:   
Post new topic   Reply to topic    www.mcselec.com Forum Index -> KokkeKat FAT-free SD card lib All times are GMT + 1 Hour
Goto page Previous  1, 2, 3
Page 3 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum