Errors with main.bas program

KokkeKat · Joined: 08 May 2011 Posts: 59 Location: Stockholm

Hi Neil

There is currently no support for deletion of file or subdirectory. While I can see the benefit from having it, I haven't found the time to develop it. If you would like to contribute, please share your code.

What you need to do is:
A restore each "record" in the entire FAT chain to 0 (so it becomes available)
B set the first byte of the (sub)directory entry to &HE5 (so it gets marked as "reusable")
C (not absolutely necessary: write 0 to every byte in the clusters that contained the file or subdirectory data)
D update the fsinfo variables to reflect the "regained" cluster space
E anything I just failed to mention

I would recommend not keeping (reusing) the directory entry, but that's your decision.

---

The file writing loop you mention could be a bug in the library. The number 16 is likely to come from the fact that each directory sector contains a maximum of 16 directory entries. (16 * 32 = 512.) This could be that the transition from the first to the second directory cluster isn't working (in case of 1-sector-per-cluster) or that the transition from the first to the second directory sector isn't working (in case of >1-sector-per-cluster). Could you please post your code so I can debug it?

It sounds like you need to format the SD card. I also recommend using a hex editor. I bought Winhex, but I'm sure there are others. Such a software would enable you to see (and change) the actual data stored on your SD card.

Kind regards

Niclas

njepsen · Joined: 13 Aug 2007 Posts: 469

Niclas,
I asked how a file is referenced back to its parent directory. Your answer "Once you have read up on FAT32, you'll know the answer to this question." is not helpful and is condescending - I have read your and Paul's and many other docs on the subject, but its not clear how this part works. I think it goes something like this: //A 32 byte directory entry resides in a sector of 16 such entries. Each entry( directory record) has an 8.3 name, atribute ( indicating file or dir, hidden etc ), cluster address of the FIRST cluster of the file and file size.( but what about when there is no file in the directory yet - how can it have a first cluster and size?) If the file occupies more than one cluster, which I think is 32k, then the NEXT cluster addr used by that file is given in the FAT record for the previously used cluster. I.e each FAT entry points at the next available cluster, even if it hasnt been used yet ( I'm not sure about this bit?). //
This still doesnt help me when I want to add a file to a subdir. Obviously I need to locate the cluster addr recorded in the directory entry, NOT the cluster that the directory entry resides at. Do I do the file write using SDstartdircluster with this addr or do I have to read the next FAT addr ( which will be the same if the file size is <32k?)

In either case Where or how i find or calculate the appropriate variable. What is the appropriate variable?

Guess I'll pass on the deleting.
I'll post the code which writes a max of 16 files shortly. Also, If I create directories in a loop, incrementing the dir name each time round the loop, it also only creates 16 directories, then overwrites the last one. Same fault?

regards
neil

njepsen · Joined: 13 Aug 2007 Posts: 469

Niclas,
here is the code that I used to continuously loop making files or directories. In either case, only 16 were created ( according to my PC)

neil

$regfile = "m328pdef.dat"
$crystal = 8000000
$framesize = 40 ' This lib only uses ~10 $hwstack - please see the application note for more information
$hwstack = 40 ' This lib only uses ~10 $hwstack - please see the application note for more information
$swstack = 40 ' This lib only uses ~10 $hwstack - please see the application note for more information
$include "KokkeKat_FAT-free_SD_lib_declarations.bas"

' '-------------------------------------------------------------------------------
'Open a TRANSMIT channel for debug
Open "comd.4:9600,8,n,1" For Output As #1
Dim Sddebug As Byte ' Only needed for the examples
Sddebug = 0 ' Only needed for the examples

Dim Filename As Integer
Dim Filenamestr As String * 10

Filename = 200
Dim S As Integer

Do

Filename = Filename + 1
Filenamestr = Str(filename)
Sdentrynames = "000 TXT"
Mid(sdentrynames , 1 , 3) = Filenamestr
Sdstatus = 0
Gosub Sdinit
Print #1 , "init done"
If Sdstatus = 0 Then
Gosub Sdinitfs
Wait 1
' Gosub Make_file
Gosub Make_directory
Loop
Else
Print #1 , "bad sdstatus after SDinitfs"
End If
End ' End program

$include "KokkeKat_FAT-free_SD_lib_code.bas"
'===============================================================================
Make_directory:
Print #1 , "New dir name is " ; Sdentrynames
Sdcreatemode = 1 ' 1=Create subdirectory
Sdyear = 2011
Sdmonth = 2
Sdday = 17
Sdhours = 00
Sdminutes = 30
Sdseconds = 32
Gosub Sdreadfsinfo
Sdstartdirclusterd = 0 ' Root dir
Gosub Sdcreatefileordir ' Find free FAT entry, find free directory entry, and save the directory entry back to the SD card
Gosub Sdwritefsinfo 'neil added
Return
'========================================================================

Make_file:
' Create and write to the file
Print #1 , "filename =" ; Sdentrynames
Sdcreatemode = 0 ' Create file
Sdstartdirclusterd = 0 'change this if writing to subdirectory?
'Sdstartdirclusterd = sdfatclusterd
Print #1 , "sdfatclusterd=" ; Sdfatclusterd
Print #1 , "sdclsterd = " ; Sdclusterd
Sdcreatemode = 0 ' i.e a file
Gosub Sdcreatefileordir ' Find free FAT entry, find free directory entry, and save the directory entry back to the SD card
Sdstatus = 0
For Sdtempw2 = 1 To 512
Sdbuffer(sdtempw2) = 0
Next Sdtempw2
Sdtempb2 = 65 ' Write "A"
Sdtempw2 = 0
While Sdclosefile = 0 And Sdtempw2 < 1024 ' 512 times (always make sure that Sdclosefile = 0 before writing)
Incr Sdtempw2
Sdbyterw = Sdtempb2
Gosub Sdwritebyte ' Save a byte to the card
Wend
Sdtempb2 = 66 ' Write "B"
Sdtempw2 = 0
While Sdclosefile = 0 And Sdtempw2 < 1024 ' 512 times (always make sure that Sdclosefile = 0 before writing)
Incr Sdtempw2
Sdbyterw = Sdtempb2
Gosub Sdwritebyte ' Save a byte to the card
Wend
Sdtempb2 = 67 ' Write "C"
Sdtempw2 = 0
While Sdclosefile = 0 And Sdtempw2 < 1024 ' 512 times (always make sure that Sdclosefile = 0 before writing)
Incr Sdtempw2
Sdbyterw = Sdtempb2
Gosub Sdwritebyte ' Save a byte to the card
Wend
Sdtempb2 = 68 ' Write "D"
Sdtempw2 = 0
While Sdclosefile = 0 And Sdtempw2 < 1024 ' 512 times (always make sure that Sdclosefile = 0 before writing)
Incr Sdtempw2
Sdbyterw = Sdtempb2
Gosub Sdwritebyte ' Save a byte to the card
Wend
Sdbyterw = 69
Sdclosefile = 0
Gosub Sdwritebyte
Gosub Sdfinalizeafterwriting ' After writing the last byte, close the file
Gosub Sdwritefsinfo ' Save the updated fsinfo variables back
Return
'=======================================================================

KokkeKat · Joined: 08 May 2011 Posts: 59 Location: Stockholm

Dear Neil

I have chosen to share some of my development with the community as a way to "pay forward" the help I have received from other people. Some of my development can be found as projects on www.avrfreaks.net and this SD library seemed fit for a proper publication as an application note at www.mcselec.com. As I want this library to be actively used and improved by contributions from other users (and I also want to get help in identifying bugs), I asked MCS Electronics to provide this support forum. Please notice that I am not an employee of MCS Electronics and that there is no commercial relationship between me and MCS Electronics.

Please understand that I do this out of my own spare time and that this support forum is the place where users can report real or suspected bugs in the library code. This is not a support forum for the FAT32 file system and it is not a FAT32 beginner's help-me-along forum. The user must be prepared to learn a lot by himself, including reading the library code for clues about how it works.

This is the reason why I have pointed out what you need to learn with regards to FAT32, but (having already spent a considerable amount of time helping you) I have not answered out-of-scope questions.

Nevertheless, in response to your summary: You are getting close: The root directory always has a volume ID. A subdirectory always has one "." entry pointing to itself and one ".." entry pointing to the directory above. I am not sure, but I think that a directory entry for a file can (according to FAT32 specs) have a null pointer, before there is any file content. My lib always allocates a cluster when the directory entry is created, so it will always point to a cluster. A directory never has a size - only files have a size. You are also almost right about the FAT chain. Each FAT entry points to the next USED cluster. The last (used) FAT entry in the chain has an EOC (End-Of-Chain) marker >=&HFFF8 or >=&H0FFFFFF8. A file or subdirectory using <= 1 cluster has an EOC marker in its (only) FAT entry.

When you want to create a file in a subdirectory, you must know beforehand which is the first cluster of this directory's cluster chain. This you find by starting at the root directory and looking up the particular subdirectory's first cluster (comparing 8.3 names). This is done iteratively (in case of multiple directory levels) until you know the first cluster of the subdirectory in which you want to create the file. If you use Sddirlist, it will be found in Sddirlistdirentry(). This value you then assign to Sdstartdirclusterd before calling Sdcreatefileordir to create the new file. Please see the code sample I posted in thread "List SUBDir". It is fairly close to what you need.

At the time of (=just before) creating a file, you don't know in which cluster it will start. As you can see in the library code for Sdcreatefileordir, it first looks up an available directory entry (extending the directory if necessary). Then it identifies the next available FAT entry (which corresponds to what becomes the first cluster of this file). You won't know the file's or subdirectory's first cluster number until the return from Sdcreatefileordir.

Thanks for posting your code. I will run and debug it in the coming week. There could be a library bug in the transition from the first (sub)directory sector to the next. In that case I will try to reply with an updated version before Saturday (but no guarantees).

Please notice that you should only initialize the SD card when you "boot up" the AVR. So, place Gosub Sdinit and Gosub Sdinitfs before the main program loop. Also, make sure that Sdentrynames is always 11 characters long. It could be that the forum text editor omits consecutive space characters (at least the leading indentations are cut out), but just to be sure: Use Sdentrynames = "000_____TXT". (If I remember correctly, the use of strings is a bit "dangerous". If you first do Mystring = "ABCDEFGH" and then Mystring = "IJK", I think the RAM location storing Mystring will actually contain "IJK[ASCII number 0]EFGH[ASCII number 0]". I am overlaying Sdentrynameb() on top of Sdentrynames, and the lib always "really" uses Sdentrynameb(). The string variable is "terminated" by a 0 value (which is why a string declaration must be one byte longer than the actual string). (Several consecutive space characters (ASCII 32) should also work, but I'm afraid it won't "print well" in the forum.)

Kind regards

Niclas

njepsen · Joined: 13 Aug 2007 Posts: 469

Niclas,
Your reply //"If you use Sddirlist, it will be found in Sddirlistdirentry(). This value you then assign to Sdstartdirclusterd before calling Sdcreatefileordir to create the new file"?? was exactly the pointer I needed. i can get the lo byte & hi byte from the array, and assign as reqd.

I know where you are coming from, and I know that you are doing this is your spare time, and I DO appreciate the help you have given me, and also realiase the enoromous effort you must have put into the project so far. However from my point of view, when i looked at the initial description of the library, I rightly or wrongly assumed that I could simply use it as a tool and drop it into my other code. This is but a small part of a large commercial effort that i am involved in and I simply dont have the time or the will to learn about the complexities of FAT32 and reading writing to SD cards -- having said that you have forced me ( which i am now grateful for) yelling and screaming down that path. However until I can read and write to directories, delete files from a directory, and handle read/write errors, i cannot progress with the rest of the project, which is quite large. To this end, if you are willing, I would like to put this on a commercail footing so you are not helping me for nothing. I would like to discuss this further with you- my email is [removed by KokkeKat for spam reasons],. or alternatievly we could talk via skype. i will respect your privacy if you say no. Thankyou for looking at the 16 file thing - thats not important right now, so there is no hurry. I was aware that I should only do SDinit once, but was too lazy to move it right now.

regards
neil

KokkeKat · Joined: 08 May 2011 Posts: 59 Location: Stockholm

Hi Neil

Thank you for understanding. I will contact you on your private e-mail and we can continue the commercial conversation in private. I have edited your posting (removing your e-mail address) for spam reasons. If you want your e-mail address to be visible to everybody, please change it back.

I am now starting to look at the 16 file example, in case there is a library bug.

Kind regards

Niclas

njepsen · Joined: 13 Aug 2007 Posts: 469

Thanks. I was intending to do the same.

KokkeKat · Joined: 08 May 2011 Posts: 59 Location: Stockholm

Hi Neil

There is indeed a library bug in Sdcreatefileordir:. Thanks for helping me find it. Please add the two lines (at about row 1400) I have highlighted below:

KokkeKat · Joined: 08 May 2011 Posts: 59 Location: Stockholm

Hi Neil

Just a quick notice concerning Windows file system buffering. In the example above, when creating so many subdirectories that it extends the directory cluster chain, it didn't seem to help that I ejected the SD card in Windows before putting it in the AVR socket. I still had to reboot my computer for Windows to care about the entries in the extended cluster (subdirectories 228-230).

Also, another thing is that unless you are going to use a fixed SD card size (the sectors per cluster concern again), your application should adjust the amount of data it writes to each file, so that it always makes good use of whatever card size you insert. Example: if you decide on a fixed file size of 4 kB (because you use a 4 GB card) and then insert an 8 GB card, it will still only be able to store roughly the same amount of data (because you waste the additional sectors of the (only) cluster each file requires. (You will still benefit to some degree from the additional space with regards to the directories' cluster usage.)

Finally, my lib should be well equipped for handling read and write errors. In case of one occuring, you might need to do the Sdinit and Sdinitfs again, but in that case, you need to make sure that you first set all relevant "system" variables to zero, so as to avoid some of them adding up each time you re-initialize.

Kind regards

Niclas