diff options
| author | cc4b05f61e2d8f77114750386c9f9a60 <cc4b05f61e2d8f7@7114750386c9f9a60> | 2023-05-11 14:38:47 +0000 |
|---|---|---|
| committer | cc4b05f61e2d8f77114750386c9f9a60 <cc4b05f61e2d8f7@7114750386c9f9a60> | 2023-05-11 14:38:47 +0000 |
| commit | f5450bfd35a6410528d124f534c2b1a958cafe51 (patch) | |
| tree | a808b12d6ad5343fabdec7b8918df6b4d844e03f /doc | |
| parent | 5ad2bb7a6ac7e97c031908d2439808a00fff6214 (diff) | |
dmsdos-0.9.2.2 addeddmsdos-0.9.2.2
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/FAQ | 215 | ||||
| -rw-r--r-- | doc/cvf-fat.doc | 4 | ||||
| -rw-r--r-- | doc/cvfinfo.doc | 372 | ||||
| -rw-r--r-- | doc/dmsdos.doc | 1152 | ||||
| -rw-r--r-- | doc/ioctl.doc | 199 | ||||
| -rw-r--r-- | doc/libdmsdos.doc | 614 | ||||
| -rw-r--r-- | doc/messages.doc | 871 | ||||
| -rw-r--r-- | doc/patent.doc | 101 | ||||
| -rw-r--r-- | doc/testing.doc | 68 | ||||
| -rw-r--r-- | doc/troubleshooting.doc | 290 |
10 files changed, 3886 insertions, 0 deletions
@@ -0,0 +1,215 @@ +This file gives answers to frequently asked questions about dmsdos. + + +What is dmsdos ? + +Dmsdos is a Linux kernel module that allows access to msdos/win95 compressed +disk partitions. It supports different doublespace, drivespace and stacker +versions. +See file dmsdos.doc for an actual list of supported dos/win95 configurations. + + + +How do I mount a compressed partition? + +You need to mount the uncompressed host partition as usual. In this +partition, you can see the compressed partiton as large file (the CVF, +Compressed Volume File). It usually has names like 'dblspace.NNN', +'drvspace.NNN' or 'stacvol.XXX'. The whole compressed partition is inside +this file. + +You need the loopback block device driver (enable the loopback device during +kernel configuration): + + * + * Additional Block Devices + * + Loopback device support (CONFIG_BLK_DEV_LOOP) [M/n/y/?] Y or M + +The loopback block device allows mounting a file as a filesystem (read some +Linux docs about this for more information). To avoid confusion, note +that this has absolutely nothing in common with the well-known network +loopback interface. + +Example: 'mount -t msdos -o loop /DOS/dblspace.001 /mnt' + +This mounts your compressed filesystem that in fact resides in the file +/DOS/dblspace.001 under /mnt. + + + +It doesn't work! + +Well, nothing simply doesn't work. It fails at a specific point and gives +some errors or warnings before failure which are usually an important hint +for finding out what exactly went wrong. Please have a look at file +troubleshooting.doc. If you still cannot solve the problem you can send an +email to the current dmsdos maintainer. The email address and a checklist +what to include in a bug report can be found in file dmsdos.doc. + + + +I have compressed partitions, but I'm using umsdos. Can I use dmsdos without +losing umsdos features? + +No problem. It should work out of the box. If you want to use umsdos style +long filenames inside the CVF, you can even mount it as type umsdos instead +of msdos. (Uhh. Use kernel 2.0.xx or get at least 2.1.94 for that). + + + +My compressed partition has long filenames under Win95. How can I see +them with dmsdos ? + +Mount as type vfat instead of msdos. + + + +Can I compress my Linux data under dmsdos? + +You can (via umsdos), but it's not recommended. + +Just to let you know: There's some other compression software available that +may be a better choice for Linux data. See file dmsdos.doc. + + + +Can I boot Linux from a compressed partition? + +No. (Older dmsdos versions supported it, but this feature has been given up +when the CVF-FAT interface was introduced. Well, the documentation always +warned ... ) + + + +Uhhh. Write access is so ssssssslow..... + +What can I say. You decided to use compressed partitions - now you are +experiencing one of their greatest disadvantages. But dmsdos has some +special tricks for you in this case. You can... (in recommended order) + + - run the dmsdos daemon which does delayed compression in the background + when the system is idle, or use the internal daemon + + - set speedup bit #2 (allocation code skips exact search) + + - set speedup bit #6 (allocation code doesn't search at all but takes + any hole free hole) (Really Not Recommended if you don't know what + you are doing). Read the comments in the documentation, please. + +See the dmsdos documentation for details and a discussion about their +advantages and disadvantages. + + + +Is it safe to use dmsdos? + +This question is something like that one: 'is it safe to drive a car?'. + +There's no warrenty. I really can't promise that there aren't any bugs in +the driver code. I think you just like to feel safe: + + - If you don't have to write to your compressed filesystems, mount + read-only. + + - Run the dos filesystem checker every time you boot dos and backup from + time to time. + + - Backup your compressed filesystem before trying to mount read-write. + + + +How do I enable/disable long filename support? + +For long filename support use a filesystem type that knows long filenames, +e.g. vfat instead of msdos. + + + +What happens if the compressed filesystem gets full during write access? + +Don't ask. That has become too complex. :( + +Well, in the worst case the same thing happens as under Dos: you get a +strange error (Dos: "sector not found") and might lose data. It is difficult +because some applications do not check the return values of write access +functions (AAARGHHH!!!). On a compressed filesystem not only cluster +allocation, but also usual write access may fail (i.e. when you replace +data that compress well by data that don't compress well and thus need more +physical space). So just keep in mind that you shouldn't fill a compressed +partition up to the last byte :) + + + +What's the difference between win95 doublespace and drivespace 3 ? + +Please note that win95 has two different drivespace programs. One of them +is included in win95 and it's in fact an old dos doublespace version. I +still refer to it as 'win95 doublespace' though M$ call it 'drivespace'. +This one is supported without restrictions. + +The other one is sold seperately from win95 in a 'M$ Plus' package (is this +still true?) M$ have called it Drivespace 3. Meanwhile it should also +be supported quite well. + +If you aren't sure, watch the filename of the large, hidden CVF. If it is +something like 'dblspace.xxx' it's the old version. If it's 'drvspace.xxx' +instead and you are very sure you created it under win95, it's Drivespace 3. +Dmsdos also tells when it detects a drivespace 3 format CVF. + +The main difference between these two versions is that drivespace 3 uses a +larger cluster size (32KB instead of 8KB) and can compress up to 2GB of data +instead of only 512KB. It has also a more advanced compression scheme +(SQ-0-0). + + + +When do I have to defragment my compressed partitions? + +Watch the dutil output (see file dmsdos.doc for a dutil description). It +displays a fragmentation value. You should keep it below 75% to feel safe. +When dutil even displays a warning about high fragmentation, you should +immediately boot Dos and defragment the CVF. + + + +Does dmsdos work with FAT32 filesystems? + +Yes. All earlier problems have been avoided by moving dmsdos one layer down +in the filesystem interface. Well, it does not support *compressed* FAT32 +filesystems. Win95 doesn't do either :) But, if some day they show up to +exist, I might be persuaded to implement it also. + + + +What about the relationship between dmsdos and software patents ? + +I'm still receiving mails about this question. At my best knowledge dmsdos +does not violate any software patents. (Well, I don't hope so, but my best +knowledge might be wrong. I'm not a lawyer.) If you are interested in +details take a look at file patent.doc. + + + +Do I have to recompile dmsdos when I upgrade my kernel ? + +The safe answer is yes. It's best to recompile *all* modules if you change +something in your kernel though not always necessary. But who knows. + +Some clever people invented a 'module version information' trick. You can +enable it during kernel configuration. This makes your modules less kernel +version dependent e.g. a module compiled under kernel 2.0.33 may also work +under kernel 2.0.34. In that case, you may share one precompiled dmsdos +module with several kernels. But please don't overuse it. You should not, +for example, use the same compiled object code of a dmsdos module for +kernel 2.0.34 and 2.1.105 - this is very likely to cause problems :) + + + +Where does the name "DMSDOS" come from ? + +It has historical reasons. The first piece of dmsdos code was written in +times of MSDos 6.2 when Doublespace became very popular. Thus, dmsdos was +meant as something like "doublespaced msdos". Today, it supports a lot of +more compressed Dos filesystems, and a better name would be "cmsdos" for +"compressed msdos". But you never change the name of a successful project :) diff --git a/doc/cvf-fat.doc b/doc/cvf-fat.doc new file mode 100644 index 0000000..ef064f2 --- /dev/null +++ b/doc/cvf-fat.doc @@ -0,0 +1,4 @@ + +Documentation for the CVF-FAT interface is part of the Linux kernel +(either directly in 2.1.xx or after applying the CVF-FAT patch for 2.0.xx). +It can be found in file linux/Documentation/filesystems/fat_cvf.txt. diff --git a/doc/cvfinfo.doc b/doc/cvfinfo.doc new file mode 100644 index 0000000..3ab66a5 --- /dev/null +++ b/doc/cvfinfo.doc @@ -0,0 +1,372 @@ +This file contains some information about the compressed filesystem layout. + + The CVF Hacker's Guide :-) + ============================== + +WARNING: This is not official M$ specs. In fact, it's a hacker's document. + I don't know M$ specs, so this file may contain incorrect + information. Use at your own risk (see the GPL for details). + +WARNING 2: Several parts of the compressed filesystem internals are still + unknown to me. If this document is inaccurate in some details, it's + because I don't know it more exactly. Feel free to add your + knowledge. + + +CVF format overview +------------------- + +version compression SPC(*) max. size +dos 6.0/6.2 doublespace DS-0-2 16 512MB +dos 6.22 drivespace JM-0-0 16 512MB +win95 doublespace/drivespace DS-0-0 16 512MB +win95 drivespace 3 JM-0-0,JM-0-1,SQ-0-0 64 2GB + + (*)=Sectors Per Cluster + +General filesystem layout +------------------------- + +Superblock (1 sector) +BITFAT (several sectors) +MDFAT (~ twice as large as FAT) +Bootblock (1 sector) +FAT (only one) (several sectors) +Root directory (some sectors) +Data area (many sectors) +Final sector (1 sector) + +There's some slack (or "reserved space") between some filesystem structures, +but I don't know what it is good for. Perhaps M$ don't know either. + +Sector counting +--------------- + +The Superblock is referred as sector 0. The rest of the sectors are counted +appropriately. + +Superblock layout +----------------- + +Byte positions are counted beginning with 0 for the first byte. Integers are +in low byte first order. Only important fields are listed here, usual dos +fields are omitted. + +Pos. 3-10: string: signature "MSDBL6.0" or "MSDSP6.0" +Pos. 45,46: *signed* integer: dcluster offset for MDFAT lookups +Pos. 36,37: first sector of MDFAT minus 1 +Pos. 17,18: number of entries in root directory +Pos. 13: sectors per cluster +Pos. 39,40: sector number of Bootblock +Pos. 14,15: sector offset of FAT start (relative to Bootblock). I.e. to + obtain the sector number of the first FAT sector add Pos. 14,15 + to Pos. 39,40. +Pos. 41,42: sector offset of root directory start (relative to Bootblock). To + obtain the sector number of the first root directory sector add + Pos. 41,42 to Pos. 39,40. +Pos. 43,44: sector offset of Data area minus 2 (relative to Bootblock). To + obtain the sector number of the first Data area sector add + Pos. 43,44 to Pos. 39,40 and finally add 2. +Pos. 51: version flag (0=dos 6.0/6.2 or win95 doublespace, 1=??, + 2=dos 6.22 drivespace, 3 or 0 ??=win95 drivespace 3) + Hint: drivespace 3 format can be recognized safely by watching + the sectors per cluster value. The version flag seems to lie + for drivespace 3. +Pos. 57-60: usually string "12 " or "16 " as the rest of "FAT12 " and + "FAT16 " (the spaces are important), but here seems to be a bug + in some doublespace versions. PLEASE IGNORE THIS VALUE, IT + SOMETIMES LIES. Use the Bootblock's value instead. +Pos. 62-63: Maximum size of the CVF in Megabytes. +Pos. 32-35: Faked total number of sectors (it is something like the real + number of sectors in the data area multiplied with the + compression ratio). This value is important because it determines + the maximum cluster number that is currently allowed for the + CVF according to this formula (don't ask me why): + + (Pos.33-35)-(Pos.22,23)-(Pos.14,15)-(Pos.17,18)/16 + max_cluster=--------------------------------------------------- + 1 + (Pos.13) + + (rounded down). Be sure not to exceed the limits due to FAT/MDFAT + size or CVF size here. Since this formula has been found by + trial and error, it may not be true in all screwy cases. + +BITFAT layout +------------- + +The BITFAT is a sector allocation map. Consider it as a list of bits each of +which represents one sector in the Data area. If a bit is set, the +appropriate sector contains data - if the bit is clear, the sector is free. + +The first bit matches the first sector in the data area (and so on). The +bits are counted *wordwise* beginning with the most significant bit of the +word (where "word" means two bytes at once, low byte first). + +So substract the number of the first data sector from the number of the data +sector you want to lookup information in the bitfat. Keep the result in +memory. Divide the resulting number by 16, round down, multiply with 2. Get +the two bytes at this position in the bitfat (counted from its beginning) +and store them as word. Now watch the least 4 bits of the previosly +memorized result - they represent the bit number (counted from the most +significant bit) in the word. This bit corresponds to the data sector. + +WARNING: The BITFAT sometimes is incorrect due to a missing system shutdown + under dos. If you want to write to the filesystem, be sure to + check (and, if necessary, repair) the BITFAT before. See below + how to do this. + +MDFAT layout +------------ + +MDFAT is organised as a stream of long integers (4 bytes, for drivespace 3: +5 bytes). The data are sector-aligned - this means for drivespace 3 that the +last two bytes of a sector are slack. Consider the bytes in usual order +(low byte first). + +The MDFAT contains additional information about a cluster: + + 3322222222221111111111 (doublespace/drivespace) + 10987654321098765432109876543210 + uchhhhllll?sssssssssssssssssssss + + 333333333322222222221111111111 (drivespace 3) + 9876543210987654321098765432109876543210 + uchhhhhhllllllf?ssssssssssssssssssssssss + +u=1: The cluster is used, u=0: the cluster is unused. In the latter case the + whole entry should be zerod. An unused cluster contains per definition + only zeros ( C notation: '\0'). This is important if a program insists + on reading unused clusters! +c=1: The cluster is not compressed, c=0: the cluster is compressed. +h: Size of decompressed cluster minus 1 (measured in units of 512 bytes). + E.g. 3 means (3+1)*512 bytes. +l: Size of compressed cluster data minus 1 (measured in units of 512 + bytes). If the cluster is not compressed according to the c bit, this + value is identical to h. +f: fragmented bit for drivespace 3. If it is set the cluster is fragmented + and needs some special treatment on read and write access. +?: Unknown. Seems to contain random garbage. +s: starting sector minus 1. I.e. if you want to read the cluster, read (l+1) + sectors beginning with sector (s+1). If the c bit is zero, the data must + be decompressed now. + Important: if the cluster on disk is shorter than the filesystem's + sectors per cluster value, the missing rest at the end has to be treated + as if it was zerod out. + +To lookup information in the MDFAT, take the cluster number, add the +dcluster offset (which may be negative!) and take the appropriate entry +counted from the beginning of the MDFAT. Don't ignore the sector alignment +for drivespace 3. + +Bootblock layout +---------------- + +Emulates normal dos filesystem super block. Most dos fields are identical +to the Superblock except for the FAT16 or FAT12 string. The FAT bitsize string +that can be found in the Bootblock is correct while the one in the +Superblock may be garbage. Take a disk viewer and compare Bootblock and +Superblock yourself. There are slight differences, but I don't know exactly +where and why. You'd better never change anything in these blocks... + +FAT layout +---------- + +No need to explain. It's the same like in a normal dos filesystem. It may be +12 or 16 bit according to the Bootblock, but *not* to the Superblock. This +seems to be a bug in doublespace - the Superblock's FAT bit size information +is sometimes wrong, so use the Bootblock's information. + +Root directory +-------------- + +The same as in a normal dos filesystem. (The root directory is never +compressed.) + +Data area +--------- + +Well, that's the actual space for the data. + +Final sector +------------ + +Contains the signature "MDR". Must not be used by data. To find it you must +know the size of the CVF file. There's no pointer in the Superblock that +points to this sector. + +Compressed clusters +------------------- + +Compressed data (when the c bit is 0 in the MDFAT entry of a cluster) are +identified by a compression header. The header consists of 4 bytes which are +at the beginning of the compressed cluster data. The headers consist of two +bytes specifying the compression scheme and two bytes version number, and +usually look like this: + +'D', 'S', 0x00, 0x02, I write it as 'DS-0-2' +'J', 'M', 0x00, 0x00 +'S', 'Q', 0x00, 0x00 + +The version number seems to be ignored though M$ claim that, for example, +'High' (JM-0-1) compresses better than 'Normal' (JM-0-0). That's nonsense +from the compressed format point of view, the format is in fact the same. +Maybe the original M$ software uses different *compression algorithms* +which may be more or less efficient, but they're not using not different +*compression schemes*. So in fact there are three schemes: DS, JM, and SQ. +DS and JM are quite similar, for a decompression algorithm see the dmsdos +or thsfs sources (both are GPL code, you may reuse it). + +As far as I know, dos 6.x versions of doublespace/drivespace never compress +directories and never cut them off (if only the first sectors of the cluster +are used, it is in fact possible to cut the cluster since the unused slack +is, per definition, to be treated as if it was zerod out). It is unknown +whether these versions can read compressed or shortened directories, but it +is sure they never compress or shorten them. So I just recommend not to do it +either. drivespace 3 usually cuts off directories and sometimes even +compresses them though compression of directories is a great performance loss. +win95 doublespace/drivespace (not drivespace 3) never cuts directories but +also compresses them sometimes. + +Fragmented clustes +------------------ + +To make things more complex, M$ have invented these strange things. +Unfortunately, they need some special treatment. + +A fragmented cluster can be recognized by watching the 'f' bit in the MDFAT. +This bit only exists in drivespace 3 format. + +The first sector of the cluster contains a fragmentation list. This list +contains entries each of which use 4 bytes. The first one is the +fragmentation count - it specifies into how many fragments the cluster is +devided. It must be > 1 and <=64. + +The following entries are pointers to fragments of data like this: + + 3322222222221111111111 + 10987654321098765432109876543210 + lllllluussssssssssssssssssssssss + +s: start sector minus 1 - the fragment begins at sector (s+1). +u: unused and zero (?) +l: sector count minus 1 - the fragment contains (l+1) sectors beginning + with sector (s+1). This means raw data if compressed. + +The first entry always points to the fragmentation list itself. I.e. +the s and l fields of the first fragmentation list entry are always the same +as the ones in the MDFAT entry. The first fragment is not restricted to +contain *only* the fragmentation list, however. + +Now it becomes slightly difficult because the data are stored differently +depending on whether the cluster is compressed or not. If the cluster is +compressed the raw (compressed) data begin immediately after the last entry +of the fragmentation list. The byte position can be calculated by multiplying +the fragmentation count with 4. Further raw data can be found in the other +fragments in order. + +If the cluster is not compressed, the (uncompressed) data begin in the +sector that follows the sector containing the fragmentation list. If the +first fragment has only the length of 1 sector the data begin in the second +fragment. Further data are in the fragments in order. + +General rules for cluster access +-------------------------------- + +I'm assuming you want to access cluster number x (x!=0 i.e. not root directory +- this one should be clear without further explanation). + +How to read cluster x from the compressed filesystem +---------------------------------------------------- + + * Get and decode the MDFAT entry for the cluster: lookup entry number + (x+dcluster). dcluster and start of the MDFAT can be obtained from the + Superblock. + + * If the MDFAT entry is unused (u bit clear), just return a cluster full of + zeros (0x00). + + * Read (l+1) sectors beginning with sector (s+1). + + * If the cluster is fragmented ... uuhhhhh ... you'd better issue an + error and encourage the user to boot win95 and defragment the drive. + Otherwise read and interpret the fragmentation list now. + + * If the data are compressed (c bit clear) decompress them. + + * If the cluster is shortened (i.e. h+1 < sectors per cluster) zero out + the rest of the cluster in memory. The sector per cluster value can be + obtained from the Superblock. + +How to write cluster x to the compressed filesystem +--------------------------------------------------- + +WARNING: Be sure you can trust your BITFAT, i.e. have it checked before. + See below how to do this. + + * Be sure to know whether the cluster may be shortened. The size in + sectors minus 1 will become the h value of the MDFAT entry later. + + * If you want, compress the data. Be sure the data really become smaller. + Determine the size of the compressed data in sectors and subtract 1 - + this will become the l value of the MDFAT entry later. If you don't + want to compress the data or the data turn out to be incompressible, + set the l to the same value as h and use the uncompressed original data. + DON'T ACTUALLY WRITE TO THE MDFAT AT THIS POINT! + + * Delete the old cluster x that may have been written earlier (see below). + + * Search for (l+1) free continuous sectors in the BITFAT. Be prepared for + failure here (i.e. if the disk is full or too fragmented). Allocate the + sectors by setting the appropriate bits in the BITFAT. Now you can create + the MDFAT entry and write it to disk - please note to subtract 1 from the + sector number when creating the s value of the MDFAT entry. Also don't + forget to set the c bit if the data are not compressed. + + * Write the (l+1) sectors to disk beginning with sector (s+1). + +How to delete cluster x in a compressed filesystem +-------------------------------------------------- + +WARNING: Be sure you can trust your BITFAT, i.e. have it checked before. + See below how to do this. + + * Get the appropriate MDFAT entry (x+dcluster). If it is unused (u bit + clear) there's nothing to do. + + * If the cluster is fragmented, scan and check the fragmentation list + and free up all the fragments. + + * Otherwise free up (l+1) sectors beginning with sector (s+1) in the BITFAT + by clearing the appropriate bits. Be sure to do a range checking before so + you don't corrupt the filesystem if there's garbage in the s field of + the MDFAT entry. + + * Zero out the MDFAT entry completely. Don't just clear the used bit. + +How to check and repair the BITFAT +---------------------------------- + +Dos seems to recalculate the BITFAT on each bootup. This points out that +even M$ programmers didn't trust it, so you shouldn't do either if you plan +to write to the compressed partition. + +It's easy. Just scan the complete MDFAT for used entries (u bit set). You +get from the l and the s values (don't forget to add 1 in each case) which +sectors are allocated. Doing this for the whole MDFAT, you get a list of +which sectors are used and which are free. Then you can compare this list to +the BITFAT. If you just keep the list in memory in the same bit encoding as +used in the real BITFAT, you can just write the complete list to disk and +replace the BITFAT by it. Uhh, yes, you may need up to 512 KB memory for +the data for this purpose... + +If you are using drivespace 3 please keep in mind that you also have to +take care of fragmented clusters (i.e. check the fragmentation bit and scan +the fragmentation list if necessary). + +Further related documents about compressed filesystems +------------------------------------------------------ + + - thsfs source (sunsite and mirrors) + - dmsdosfs source (sunsite and mirrors) + - Bill Gates' secret drawers + - Murphy's law diff --git a/doc/dmsdos.doc b/doc/dmsdos.doc new file mode 100644 index 0000000..50ac6fe --- /dev/null +++ b/doc/dmsdos.doc @@ -0,0 +1,1152 @@ +dmsdos.doc + +This is the main documentation for the DMSDOS CVF module. 01MAR1999 + +------------------------------------------------------------------------------ + + This is version 0.9.2.1. + +This version has been designed for Linux 2.2.2, but it should run under all +2.2.x kernels. It also works with kernels from 2.0.29 to 2.0.36. It very +likely also works with older ones, but it has at least been reported to fail +with 2.0.0. For the 2.1.xx series you need at least 2.1.80 though it is +recommended to use 2.1.94 or newer. + +Installation notes see file ../INSTALL.TXT. + + +Contents: + +1. DMSDOS capabilities +2. Restrictions +3. Mount options +4. Kernel messages [moved to file messages.doc] +5. Installation [moved to file ../INSTALL.TXT] +6. Troubleshooting [moved to file troubleshooting.doc] +7. Defragment procedures +8. How to contribute to dmsdos development +9. The external dmsdos utility 'dutil' +10. The dmsdos daemon 'dmsdosd' +11. The dmsdos library 'libdmsdos.a' +12. The dmsdos filesystem checker 'dmsdosfsck' +13. The dmsdos Midnight Commander external filesystem utility 'mcdmsdos' +14. Authors and email addresses + +------------------------------------------------------------------------------ + + +1. DMSDOS capabilities +------------------------------------------------------------------------------ + + * 'mount' + - doublespace/drivespace compressed msdos 6.x partitions (read-write), + - doublespace and drivespace 3 compressed win95 partitions (read-write), + - and stacker 3 and 4 compressed partitions (read-write) + under Linux via the loopback block device + * supports compressed floppies (yes, and even cdroms[**]) + * can use umsdos extensions in compressed partitions + * when writing, compression level may be selected by user + * supports standard text conversion like the msdos driver + * simple internal fs check on mount (can be switched off) + * dmsdos daemon for compressing data in the background + * works with all drivers that base on the fat driver, also with FAT32 + * stand-alone dmsdos library for CVF related programming + * stand-alone dmsdosfsck (experimental) + * various other utilities + + [**] No, don't really think of burning a CVF onto a cdrom. Dos cannot read + it unless you copy it to your hard disk. REALLY BAD. Linux+dmsdos can. + +In detail: + + Dos compressed partitions are always stored inside an uncompressed host + partition as large files (CVFs, Compressed Volume Files). In order to + access them from Linux, you need + * the CVF-FAT interface, which is present in kernels 2.1.80 and newer + and must be patched into the 2.0.xx kernel series - the patches are + included in this dmsdos release + * and the loopback block device. + The latter lets you mount a file as a filesystem. You must compile the + loopback block device into your kernel or as a loadable module in order + to use this feature. + + This version works with compressed hard disks and with compressed floppies. + It has been tested successfully with MS-DOS 6.2 Doublespace and MS-DOS 6.22 + Drivespace compressed formats. win95 doublespace is supported, also the + drivespace 3 format from the Micro$ Plus package (rumors tell that this + package has been integrated into Win95OSR2). Stacker version 3 and 4 + are supported too. Dmsdos can even mount CVFs from cdrom (Dos can't). + + Francesco Potorti([email protected]) has reported that dmsdos + also mounts Superstore compressed filesystems. I can't test this as I + don't have Superstore. They are recognized as old Dos 6.0 Doublespace + format. I'm interested in further reports about this. I still can't + believe that M$ simply copied (bought?) Superstore and renamed it to + Doublespace without further modifications... + + Since version 0.9.0, dmsdos runs *below* the msdos/fat filesystem. This + makes it compatible with all fat based filesystems including FAT32, NLS + and other things :) + + Dmsdos does a lot of internal caching in order to reach a reasonable speed + with a compressed (and really badly organised, but simple) filesystem. As + a side effect, this cache may eat a lot of system memory (you can change the + size limit). + + Dmsdos was initially inspired by the THS filesystem (once found under + ftp://sunsite.unc.edu/pub/Linux/system/Filesystems/dosfs/thsfs.tgz) + written 1994 by Thomas Scheuermann (email address out of date), which was + the first filesystem available under Linux that could read some CVFs. + + When mounting a compressed filesystem, the dmsdos driver does some quick + and simple consistency checking. If it finds a serious filesystem error + that might cause further damage on write access, it sets the compressed + filesystem to read-only mode. The filesystem check can be switched off if + you don't like it. + + The dmsdos daemon included in this release can speed up dmsdos write + access on systems that usually run at high processor load using delayed + compression in the background when the system is idle. I recommend to + use it if you are going to write much to compressed partitions from Linux + and need the processor time mostly for other tasks. The daemon also has a + significant effect on dmsdos memory management which might be important + for systems with little memory (well yes, dmsdos *is* a memory monster + that can eat up to 2MB of system memory for caching in the default + configuration). See chapter 'The dmsdos daemon' for details. + + The dmsdos library is a stand-alone tool for raw CVF programming. + 'Stand-alone' means that it works independently from the dmsdos kernel + module, on *raw* CVFs (i.e. those large files lying around in an + uncompressed host partition). It's the same source code, though, just + compiled in a slightly different way. Meanwhile, it compiles even under + Windows :) + + +2. Restrictions +------------------------------------------------------------------------------ + + (See also file BUGS for a list of known bugs.) + + Performance degradiation (well, not really bad in most cases...) + ------------------------ + + Access to compressed filesystems slows down your system. This is a fact + that results from additional time needed to decompress or compress the + data and to maintain additional filesystem structures. It's an untrue + rumor (sometimes spread by compressed filesystem software sellers) that + compressed filesystems speeded up slow ISA bus systems because of the + smaller amount of data to be transferred through the slow ISA bus - + there's too much overhead otherwhere. + + Though most time-consuming actions like compression and finding free + sectors have been greatly improved since dmsdos 0.8.0, there are still + sudden system freezes reported. Reasons for this include (in rising order): + + * Decompressing the data takes some time, but that is supposed to be + minimal. You will probably not notice it. + + * Badly written programs issuing many redundant reads. Such programs + kill performance on every filesystem, but it's extremely bad under + dmsdos because of the filesystem layout. + + * Large files, especially accessing them near their end. Again the + filesystem layout is the cause. (How do you find the end of a file + in a fat filesystem? Right, read the fat. And if the file is large + there are tons of fat access necessary to find the end.) + + * Huge directories. It's again the fat access problem like above. (In + 2.1/2.2 kernels it's faster due to the gain from dentry caching.) + + * Compression on write access. Compression speed depends heavily on + the data to compress. In general you can say, the better the data + compress the faster the compression. + + * Compressing incompressible data (on write access). This only wastes + time since the driver throws the compressed data away and uses the + uncompressed data if the compressed data turn out to be in fact + larger. The only thing you can do against this is disabling + compression if you know the data you want to write don't compress + any more (e.g. when writing already compressed files). + + * Highly fragmented (at internal MDFAT level) filesystems on write + access. This puts the search algorithm that searches for free space + in a long loop finding a fitting hole for data to be written. This is + probably the most important reason for sudden system freezes. It's + also the most annoying one since there's really nothing to do against + it. This kind of fragmentation is normal to every compressed + filesystem and cannot be avoided. The more you write the worse it + becomes. (See below for details on that kind of fragmentation. Note + that it's very different from that FAT level fragmentation known from + FAT based filesystems.) + + Compression and free sector finding may be fine-tuned with some of + the dmsdos options. This is not really explained here, see mount + options cf:xx and speedup:xx. + + Dmsdos never freezes the system totally, it periodically unfreezes even on + extremely hard disk access. You *can* login into another console and you + *can* enter commands there. No keystroke gets lost though you may not see + a reaction on the console immediately, but after some seconds you should + see something. Just be warned and prepared :) + + Compressed formats + ------------------ + + The scheme I named the compression methods is simply this: Take the first + four bytes of a compressed cluster. For the DS-0-2 method, for example, + they are 'D','S',0x00,0x02. + + Format: Used by: Readable: Writable: + DS-0-0 win95 doublespace yes yes + DS-0-1 never reported to exist works in theory :) + DS-0-2 msdos 6.2 doublespace yes yes + JM-0-0 msdos 6.22 drivespace yes yes + and win95 drivespace 3 yes yes + JM-0-1 win95 drivespace 3 'High' yes yes + SQ-0-0 win95 drivespace 3 'Ultra' yes yes + uncompressed all versions yes yes + (no header) Stacker 3 yes yes + 0x81-0 Stacker 4 yes yes + + Yes, meanwhile all currently known methods are supported (hmm, why is this + still listed under restrictions...) + + Fragmented clusters in drivespace 3 volumes + ------------------------------------------- + + They can be read since dmsdos 0.8.1pl5. They can be written since + 0.9.0pl10, but writing fragmented clusters is supposed extremely + experimental. If dmsdos needs to rewrite data that are stored in a + fragmented cluster it first deletes the old data and saves the new + data in the usual linear order. Fragmentation writes are only done + if the data do not fit on the disk in an unfragmented way. Currently, + dmsdos becomes VERY noisy when it writes fragmented clusters... + This is considered experimental and needs further testing. You can + currently switch off this feature by setting speedup bit#8 (this is + a mount option, see below). + + Fragmented and suballocated clusters in Stacker volumes + ------------------------------------------------------- + + Of course, they can be read. But currently, they cannot be written. + If dmsdos needs to rewrite data that are stored in a fragmented or + suballocated cluster it currently deletes the old data and saves the new + data in the usual linear order. Fragmented write may come soon, + suballocated write is more complex and currently considered not very + important for dmsdos. We'll see... + + Swapping to a file on the compressed partition + ---------------------------------------------- + + Just No. It doesn't work. Don't bother me with ideas of swapping to a + compressed ram disk which would be a memory compressor... bah! + That's all gibberish, crap, and dogsh*t. I won't write code to support + this. You know very well that swapping has to be extremely fast, so use + a partition on your fastest disk for this purpose. Not a swapfile, and + never in a fat based filesystem (just imagine what a filesystem driver + has to do in order to access the *end* of a large file... yeah, tons + of fat access...) + + attention, DOSEmu users + ----------------------- + + You may have to unmount all fat based partitions before running + DOSEmu depending on your configuration. If DOSEmu is configured to use + wholedisk or partition access (this is often the case to let DOSEmu access + compressed partitions) there's a risk of destroying your compressed + partitions or crashing your system because of confused drivers. + + Let me explain why. Dmsdos does a lot of internal caching, but cache + contents won't be up to date if DOSEmu directly modifies the disk. This + confuses dmsdos and may mess up your data. On the other hand, dos + dblspace/drvspace.bin also has a kind of caching mechanism that gets + confused by the Linux dmsdos driver's write access. Confusion even + occurs if one party mounts read-only and the other mounts read-write. + + Note that it is always safe to redirect the compressed partitions with + lredir or emufs.sys. Refer to this table for configurations that are safe + (marked with +) and which are dangerous (marked with -): + + DOSEmu: Linux fat-based filesystems: + -------------- -------------------------------- + not mounted mounted + mounted ro rw + wholedisk ro + + - + wholedisk rw + - - + partition ro + + - + partition rw + - - + lredir/emufs.sys ro/rw + + + + Memory mapping + -------------- + + *Should* work. :^) + + Underlying dos filesystem limitations + ------------------------------------- + + None. Really, they're gone since the CVF-FAT interface is used. + + Umsdos users warning + -------------------- + + If you want to use umsdos-style long filenames in compressed filesystems, + use kernel 2.0.xx or get at least kernel 2.1.94. + + Win98 compatibility + ------------------- + + HEY! Windows People ! + Everyone uses it in the Windows World(TM) :)) Let me know whether + it works, please. (I refuse to buy Win98 just for such a test.) + Or have you all switched to WinNT yet ? + + WinNT compatibility + ------------------- + + No, WinNT does not support compressed FAT partitions. But... + :)) + There's meanwhile a small chance to port dmsdos to WinNT. This is + somewhat interesting because neither Doublespace nor Stacker runs under + WinNT. But I can't do it myself because of lack of knowledge :( For more + information, see file PORT_TO_WIN32. The trick is hacking in the dmsdos + library so it compiles under that operating system... I just managed to + compile one dmsdos utility, mcdmsdos, under WinNT... + + Last but least + -------------- + + Dmsdos has been written for dos/win95 compatibility. It's just a relict + from the time when I programmed some software that had to run both under + dos and Linux and I had to compress my drive because hard disk space was + rather expensive. Times and prices have changed a lot since then (at least + in the country where I live), but dmsdos has still been improved for fun. + + If you don't need dos/win95 compatibility, i.e. if you want to compress + Linux-only data, let me say: DON'T DO IT. Clean up your disk, or buy a + larger one, but don't compress it. Really. Compress single files (e.g. + manpages) with gzip. Throw away the crap you don't need. Shrink your + dos partition or remove it completely :) + + If you still want to compress Linux data, dmsdos may not be the software + you want. The msdos filesystem is extremely unsuited for fast disk access, + umsdos suffers from it too. The CVF structure msdos uses is even more + unsuited, and the time required by msdos online compression may kill the + rest of Linux' performance on these filesystems. + + Just to let you know about some other online compression software for Linux: + There's a package 'double' available on sunsite for Linux online + compression. There are also compression patches for the ext2 filesystem + available under the name 'hufs' and 'e2compr'. I've also heard about a + 'zlibc' project which lets you access gzip compressed files as if they were + uncompressed - it works by a patched libc. Look at common Linux ftp sites. + + Like dmsdos, all those packages may have their specific advantages and + disadvantages. Some are said to be not 100% stable, but who can claim that + his software is really 100% stable :-) Please don't flame at me if the + packages don't do what you want - I haven't tested them and I don't know + much about their qualilty. + + +3. Mount options +------------------------------------------------------------------------------ + +CVF related mount options are usually surrounded by the CVF-FAT interface +options. These are "cvf_format=xxx" and "cvf_options=yyy". The string "xxx" +can be "dblspace" for a doublespace or drivespace CVF and "stacker" for a +stacker CVF. For more information about the "cvf_format=xxx" option, +especially in cooperation with kerneld and automatic module loading, see +the dmsdos installation instructions (file INSTALL.TXT) and the generic +CVF-FAT documentation (file linux/Documentation/filesystems/fat_cvf.txt). + +When we speak about dmsdos mount options, we usually mean the string "yyy" in +"cvf_options=yyy". That's what this chapter describes. + +Summary: +-------- + +The dmsdos driver currently recognizes the following options: + +comp:xxx, cf:xxx, bitfaterrs:xxx, loglevel:xxx, speedup:xxx + +(for backwards compatibility also comp=xxx, cf=xxx, bitfaterrs=xxx and +loglevel=xxx, speedup=xxx are accepted) + +The xxx has to be replaced by an appropriate parameter, of course. All +options have to be specified separated by plus signs (not commas) in a list +in the mount command: +'mount -t msdos -o cvf_options=option1+option2+option3 ...'. +Consider the string 'cvf_options=option1+option2+option3' as *one* FAT +driver option! For backwards compatibility, also a dot (.) is recognized +as option separator symbol. + + comp:xxx + The comp option selects which compression method to use for compressing + files when they are written to a compressed partition. The default value + (which is used if no comp option is specified) is hard-coded to guess. + + Don't get confused by this list. Dmsdos usually determines automatically + the right and, if there's a choice, the best compression for you (by + scanning the filesystem and analysing the compressed data written by + dos/win95). But if you explicitely tell dmsdos to use a specific + compression, the driver does what you tell it to do. So be careful. + + comp:no Selects no compression for write access i.e. all files written + to a compressed partition are stored in fact uncompressed. + This option may speed up write access, but doesn't make much + sense on a compressed filesystem (except for debugging). + comp:ds00 Selects DS-0-0 compressed format for write access. This is + win95 native format if you don't have drivespace 3. This + format may also be used for Stacker 3 and 4 (though not + recommended for them as it's not the native Stacker format). + comp:ds01 Obsolete. DS-0-1 has never been reported to exist. So what. + comp:ds02 Selects DS-0-2 compressed format for write access. This is + dos 6.0-6.2 (*not* 6.22) format. Under dmsdos, it is rather + identical to DS-0-0 except for the compression header. + comp:jm00 Selects JM-0-0 compressed format for write access. This is + dos 6.22 format and win95 drivespace 3 'Normal' format. + comp:jm01 Selects JM-0-1 compressed format for write access. This is + win95 drivespace 3 'High' format. Note that it is rather + identical to JM-0-0 in dmsdos (but not in win95). + comp:sq00 Selects SQ-0-0 compressed format for write access. This is + win95 drivespace 3 'Ultra' format. Only drivespace 3 is + known to be able to handle this format. + comp:sd3 Selects SD-3 compressed format for write access. This is + Stacker 3 format. It may be used for Stacker 3 and 4. + comp:sd4 Selects SD-4 compressed format for write access. This is + Stacker 4 standard format. It may only be used for Stacker 4. + comp:guess Tries to find out automatically which compression method + dos used to create the compressed partition and selects the + appropriate format. If guessing fails, no compression is used + and a message is printed in the syslog. + + WARNING: You *can* specify senseless options in the command line, for + example, mount a doublespace drive and select stacker compression. + You may end up in a mess in that case. + + NOTE: It's not true that JM-0-1 (Drivespace 3 'High') compresses better + than JM-0-0 (Drivespace 3 'Normal'), at least under dmsdos. Set the + compression effectivity with the cf:xx option. In fact, dmsdos uses the + same routine to compress DS-0-0, DS-0-1, DS-0-2, JM-0-0 and JM-0-1 + (they are almost equal, so similar that only, for example, the meaning + of one bit is different or some constant offsets differ). It's just + true that SQ-0-0 is more powerful than the DS and JM formats, and also + SD-4 is more powerful than SD-3. Also, one can say that SD-3 has + something in common with the DS and JM formats. Yes, SD-4 and SQ-0-0 + have some similarities, too, but I really can't say which one is best. + + cf:xx (where xx is a decimal number from 1 to 12) + Selects the compression effectivity and speed. Since compression + effectivity always runs in concurrence to speed, try some values and + watch what happens. 1 is fastest but least efficient compression, 12 is + most efficient but slowest compression. The default value can be selected + before compiling by 'make config' (note that the value minus 1 must + be specified there, i.e. the range is from 0 to 11 there). + + If the external dmsdos daemon is running, this option doesn't have an + effect because the daemon doesn't care about this option - it reads the + compression level from its command line instead. + + bitfaterrs:xxx + Selects what to do with inconsistencies in the internal filesystem + sector allocation table. Default is setting the filesystem read-only. + + bitfaterrs:repair + Repair BITFAT mismatches if there are any when mounting the + filesystem read-write. This verifies the BITFAT and corrects + allocation errors. It is highly important for write access + that the BITFAT has no errors. For read-only access the + BITFAT is not needed. In read-only mode, this option is + ignored. This option may be dangerous in case dmsdos has + not recognized your filesystem correctly. So try without it + the first time until you can be sure. + bitfaterrs:ignore + Ignores BITFAT mismatches. This is dangerous and can cause + awful MDFAT level crosslinks as well as complete data loss. + Use this setting only if you know what you are doing. + bitfaterrs:setro + Sets the filesystem to read-only mode if BITFAT mismatches + are detected. This is safe and also the default behaviour. + bitfaterrs:nocheck + Don't check the filesystem's internal tables when mounting. + This speeds up the mount process a lot. This option + is *not* recommended, of course, unless you are very sure + there are no errors in the filesystem. + + WARNING: Generally, BITFAT mismatches are *severe* filesystem errors. + You will destroy your data if you write to a filesystem that contains + BITFAT mismatches. Note that Dos seems to check and repair the BITFAT + automatically and silently on each bootup, so just booting into Dos may + repair those errors. + + loglevel:xxx + Sets the driver's loglevel to xxx (a decimal or, if preceeded by 0x, + hexadecimal number). The number is a 32 bit field. Each bit represents a + family of messages that will be logged when the appropriate bit is set. + See file dmsdos.h for details about the message families (watch out for + LOG_SomeThing defines and what bits they use). The meanings of the bits + are not documented here since they may differ from version to version. + + WARNING: You are strongly encouraged to increase your kernel's log buffer + size to at least 64KB (it's in file linux/kernel/printk.c: + '#define LOG_BUF_LEN size_in_bytes', please note that the size *must* be + a power of 2) - otherwise you are likely to lose messages or receive even + complete garbage due to log buffer overruns. + + speedup:xxx + Sets the driver's speedup flags to xxx (note: these flags are always the + same for all CVFs). xxx is a decimal or (if preceeded by 0x) hexadecimal + value. The meanings of these bits are listed below. The speedup is active + when the appropriate bit is set. + + WARNING: You should not use this option or change the default value + unless you know exactly what you are doing. + + *** Less speedup than the default may result in painfully + sluggish filesystem access. + More speedup than the default may cause dangerous side + effects. + + However, speedup may have to be selectively disabled in order to hunt + bugs :) + + bit #0: Leave directories uncompressed + Never compress directories for drivespace 3 and stacker + (others don't support compressed directories at all). + WARNING: Directories are accessed very often, so it's + best not to compress them. Usually set this bit. + + bit #1: Leave umsdos EMD file uncompressed + This is only for umsdos upon dmsdos: never compress the + --linux-.--- special file (it contains extended directory + information). + WARNING: This file is even more often written than a + directory since it contains the access times (the + directory contains only the modification times). You will + regret compressing this file, even if you own the fastest + computer of the world. Don't ask, set this bit. + + bit #2: Skip exact search on BITFAT allocation + Search faster but less carefully for free space in bitfat + at the cost of more fragmentation. This bit is for + sector allocation code. If you set this bit allocation on + large CVFs is faster but also causes a little more + fragmentation. On the other hand, searching more carefully + leads to sudden system freezes for up to some seconds on + large partitions. Set this bit if you cannot tolerate + them. Usually this bit is cleared. + + bit #3: Fast unmount + Write dirty buffers on unmount immediately without + compressing them before. This switch defines what happens + with unwritten dirty clusters that are in the cache when + the filesystem is unmounted. If the bit is clear it means + write them back, but compress them before. If it is set, + it means write them back without compression. + WARNING: COMPRESSION TAKES SOME TIME, YOU'LL NOTICE IT. + If you are prepared to wait even some minutes (on an old + 386SX16 - on a P100 this should be max. one second) on + unmount you can clear this bit. Usually it is set. + + bit #4: Enable write-back cluster caching (instead of write-through) + If this bit is set the data in the cluster + cache that have been changed by an application and have to + be written back to disk are not written back immediately - + they are kept some time in memory just in order to save + disk access when the application again changes the data. + There's a small risk of data loss if the system happens to + crash before the data are written back - but since your + operating system is Linux a crash is rather unlikely :) + This bit is usually set. When hunting bugs, it should be + cleared since it may prevent finding a way to reproduce + a bug. + + bit #5: Enable cluster read-ahead + If this bit is set, this causes the driver to initiate a + low-level disk access for some data when it thinks they + are most likely needed in a short time later. The trick is + that the driver doesn't wait for the disk access to + finish. So the disk can position its head (and so on) + while the system can do something else. Most likely the + disk access has finished when the data are actually needed + - this saves some time we otherwise had to wait. Well, at + least this is the nice idea of read-ahead. However, since + read-ahead relies on a prediction, there may be situations + when it is useless or even a loss. + This bit is usually set. When hunting bugs, it should be + cleared since it may prevent finding a way to reproduce + a bug. + + bit #6: Fast BITFAT allocation + Switch to very fast sector allocation. This speeds up + bitfat allocation because the search algorithm that tries + to avoid fragmentation is simply switched off. BE WARNED, + it causes *much* fragmentation in very short time. The + "filesystem almost full" warning is suppressed. This + switch is meant as a last resort if you cannot tolerate + system slowdowns at all. Don't fill the compressed + filesystem up to more than 3/4 when this switch is set. + Write access may fail horribly and cause data loss due to + too high fragmentation. + This bit is usually cleared. If you are unsure about the + dangers let it cleared. Only set it if you really know + what you are doing. + + bit #7: Use daemon for background conpression if present + Use the daemon for delayed compression in the background. + This bit is for users of the internal daemon since the + internal daemon cannot be disabled by simply killing it. + So clearing this bit is another way to disable the daemon. + This bit is usually set. When hunting bugs, it should be + cleared since it may prevent finding a way to reproduce + a bug. *** This bit only affects daemon *compression*. + Daemon-controlled memory management still works + regardless of this bit. + + bit #8: Avoid fragmented writes + Speedup Bit#8 controls what to do when the filesystem is + too fragmented for normal write access. Usually all data + for one cluster are stored in contiguous sectors, but if + the filesystem is too fragmented there may not be a 'free + hole' that is large enough for the data. Speedup Bit#8 + controls what to do in that case. If this bit is set + dmsdos just returns a 'no space left on device' error and + refuses to write to the CVF. + Drivespace 3 and Stacker know a hack for that situation: + they allow storing the data of one cluster in several + fragments on the disk. If the bit is clear, the driver + tries to write the data in several fragments. Be warned, + this makes future filesystem access much slower as it + needs more disk head movements to read fragmented + clusters. + *** Note that for Stacker fragmented writes are currently + not implemented. Doublespace and drivespace (version + <=2) do not support this at all. So the bit is + ignored for them. + + Default speedup (187) is reasonable for normal read and moderate write + access. For high-traffic write access you might want to set additionally + bit #2 (speedup:191) or even bit #6 (speedup:255). On quite full + or highly fragmented CVFs this makes a *great* difference, but it's up + to you to decide between safety and speed. + +Note: + You can use the external utility dutil to setup or change dmsdos options + later. Look at the following example commands: + + dutil /mnt setcomp xxx (for changing the comp:xxx option), + dutil /mnt setcf xx (for cf:xx option), + dutil /mnt setspeedup xxx (for speedup:xxx option) and + dutil /mnt setloglevel xxx (for loglevel:xxx option). + + Just replace the directory name '/mnt' with your mount point in the + examples. Besides, you must be root to change dmsdos options via dutil. + The bitfaterrs:xxx option cannot be changed with dutil since it only + makes sense at mount time :) + + The string 'option1+option2+option3...' in the FAT mount option + 'cvf_options=option1+option2+option3...' has a hard coded limit of 100 + characters (this is a limitation of the CVF-FAT interface). The string + is silently cut down if it is too long. Don't worry, you won't manage + to break the limit without specifying senseless parameters :) + + For those who want to know everything exactly, the CVF-FAT interface + has a further FAT driver option (*not* dmsdos option): cvf_format=xxx. + The parameters that the dmsdos module allows for 'xxx' are 'dblspace' + (for all doublespace and drivespace versions) and 'stacker' (for all + stacker versions). Usually you don't care about them since the formats + are detected automatically. They may, however, be useful in order to + trigger kerneld to load the dmsdos module on demand. For details please + look at the dmsdos installation instructions (file INSTALL.TXT) and the + generic CVF-FAT interface documentation (file + linux/Documentation/filesystems/fat_cvf.txt). + + And now a warning: If you want to mount a CVF *be sure* to have the + dmsdos module loaded. The plain FAT driver also mounts some CVFs without + complaining (but begins to scream loudly when you, for example, do + a 'ls' on the mountpoint). As some CVFs look very similar to normal, + uncompressed filesystems, the FAT driver has no chance to detect a + CVF at mount time (and e.g. print an error) when the dmsdos module is + not loaded. + + *** If you always specify a "cvf_format=something" option you never + run into that problem. + +Some simple examples: + 1. Your msdos partition that contains CVFs is on /dev/hda1. You want to + mount it under /DOS and the CVF under /mnt. + mount -t msdos /dev/hda1 /DOS + mount -t msdos -o loop /DOS/dblspace.001 /mnt + 2. Like above, but you need umsdos support. + mount -t umsdos /dev/hda1 /DOS + mount -t msdos -o loop /DOS/dblspace.001 /mnt + 3. Like above, but you also have umsdos style long filenames in your CVF + mount -t umsdos /dev/hda1 /DOS + mount -t umsdos -o loop /DOS/dblspace.001 /mnt + 4. Like 1., but you need Win95 long filename support *only inside* the + compressed partition. + mount -t msdos /dev/hda1 /DOS + mount -t vfat -o loop /DOS/dblspace.001 /mnt + 5. Like 1., but you want to feel safe and refuse write access to the + compressed partitions only (but the uncompressed host should be writable). + mount -t msdos /dev/hda1 /DOS + mount -t msdos -o ro,loop /DOS/dblspace.001 /mnt + 6. Like 4., but you need Win95 long filenames also in the uncompressed host + partition + mount -t vfat /dev/hda1 /DOS + mount -t vfat -o loop /DOS/dblspace.001 /mnt + 7. You have managed to burn a compressed filesystem onto a CD (This is + quite easy - put the large file 'dblspace.001' into an iso9660 + filesystem and burn it. Be warned, dos can't mount it from CD. Uh...) + mount -t iso9660 /dev/cdrom /cdrom + mount -t msdos -o loop /cdrom/dblspace.001 /mnt + 8. Like 1., but the CVF has been umssynced and contains umsdos long + filenames. + mount -t msdos /dev/hda1 /DOS + mount -t umsdos -o loop /DOS/dblspace.001 /mnt + +Some complex examples (I didn't break the long lines herein). The option +strings might look strange but they are correct: + 9. Like 1, but you want the driver to repair bitfat errors automatically. + mount -t msdos /dev/hda1 /DOS + mount -t msdos -o loop,cvf_options=bitfaterrs:repair /DOS/dblspace.001 /mnt + 10. Like 1, but you need more speedup due to high-traffic write access + (well, please note that this is not recommended, but it works): + mount -t msdos /dev/hda1 /DOS + mount -t msdos -o loop,cvf_options=speedup:255 /DOS/dblspace.001 /mnt + 11. Like 9 and 10 (repair bitfat errors and more speedup): + mount -t msdos /dev/hda1 /DOS + mount -t msdos -o loop,cvf_options=bitfaterrs:repair+speedup:255 /DOS/dblspace.001 /mnt + 12. Like 1, but disable write access, ignore bitfat error and do debug + logging (be careful, this causes tons of debug messages): + mount -t msdos /dev/hda1 /DOS + mount -t msdos -o loop,ro,cvf_options=bitfaterrs:ignore+loglevel:0xffffffff /DOS/dblspace.001 /mnt + +Mounting via /etc/fstab: + + This may be a bit tricky. A typical fstab entry for a CVF looks like this: + + /DOS/drvspace.001 /DOSF msdos loop 1 0 + /dev/hda1 /DOS msdos defaults 1 0 + + Be sure to keep the sixth field (the 'pass number') zero to prevent fsck + from checking the CVF (it does not know about CVFs and very likely fails). + Also the filesystems in /etc/fstab seem to be mounted in reverse order, + but that may depend on your mount or libc version. Try to reorder the + lines it it doesn't work. + + +4. Kernel messages +------------------------------------------------------------------------------ + +This section has moved to file messages.doc. + + +5. Installation +------------------------------------------------------------------------------ + +This section has moved to file INSTALL.TXT in the main dmsdos directory. + + +6. Troubleshooting +------------------------------------------------------------------------------ + +This section has moved to file troubleshooting.doc. + + +7. Defragment procedures +------------------------------------------------------------------------------ + +There are three possibilities for fragmentation of a compressed partition: + - Msdos FAT level: The CVF itself is a file in a msdos partition that + might be broken into several fragments. + - Internal FAT level: The files stored in the CVF are fragmented in a + similar way like FAT level fragmentation on a real dos partition. Unlike + in a real msdos partition, filesystem access does *not* suffer from this. + There is no need for defragmenting a CVF at this level (except that you + want to make the CVF smaller - then it may be necessary). + - Internal MDFAT level: Worst fragmentation that can happen in a CVF. All + write access (not only file creation and deletion) causes MDFAT level + fragmentation. If a CVF is too fragmented at this level, write access + even fails as if the disk was full. Defragmenting a CVF usually refers + to this kind of fragmentation, which is very different from FAT level + fragmentation. + +How to defragment a CVF at internal MDFAT level see your dos/win95 or +stacker documentation. Dmsdos does not support CVF maintainance. + + +8. How to contribute to dmsdos development +------------------------------------------------------------------------------ + +First of all, thanks a lot to all alpha testers and all who have sent problem +reports, found bugs in the sources or documentation, suggested solutions to +some problems and asked questions due to unclear documentation. Without your +help dmsdos would not have become what it is. + +Also many thanks to all who simply mailed they liked dmsdos. + +At this place, I must also say many thanks to Gordon (VFAT fs) and Matija +(UMSDOS fs) for the help to make dmsdos work with the standard 2.1.xx +kernels. + +If you have a problem that is not covered in the documentation please email +me. But, *please*, be sure to read the file 'troubleshooting.doc' before. +If it's not mentioned there, you may have uncovered a bug that +should be fixed. Please watch your kernel log (/var/log/messages) for +strange dmsdos messages too. Please include the following information in your +mail message: + +- Linux kernel version, dmsdos version (see kernel messages). +- If it doesn't compile, gcc version, as version, libc version, make version. +- MS-DOS version you a) used to create the compressed partition, b) you + currently use. +- Dmsdos kernel messages that are produced when the problem occurs. +- Information about your disk hardware (hardware sector size is important - + there are some hacks for hardware sector sizes != 512 bytes in the code) +- Size of the real (uncompressed) dos partition. +- dutil output (run it on the dmsdos CVF directory that causes problems). +- Everything else, of course, you think I need to know about the problem. +- A valid email address (I wouldn't write this if I hadn't received some + mails I couldn't reply to because the return address was rejected). + + +9. The external dmsdos utility 'dutil' +------------------------------------------------------------------------------ + +(For a summary, take a look at the dutil manual page (file src/dutil.1).) + +Run dutil on a dmsdos mount point to obtain additional information about the +compressed partition. Example: 'dutil /mnt'. This utility has +some additional debugging options. These are not described here since they +are for debugging only. Run dutil with debug options only when you are asked +by the dmsdos author/maintainer in a response to any problem report. + +Note: Debugging options require running dutil as root. Other users can only +see the general filesystem information dutil prints on the screen. See file +dblspace_ioctl.c for details on what can be done only by root. + +You can use the external utility dutil to setup or change dmsdos options +later. Use 'dutil /mnt setcomp xxx' (comp:xxx option) and +'dutil /mnt setcf xx' (cf:xx option). Note to replace the directory name +'/mnt' with yours. This also works: 'dutil /mnt setspeedup xxx' +(speedup:xxx option) and 'dutil /mnt setloglevel xxx' (loglevel:xxx option). +The dmsdos options can only be changed by root. + +To perform a simple filesystem check, use the command +'dutil /mnt checkfs'. This fs check doesn't do any repairs, but +immediately reports some fs inconsistencies. There shouldn't be any unless +the driver complained already at mount time (see kernel log). If there are +errors and the dmsdos driver didn't find them at mount time, this is a +serious dmsdos bug (it implies that data have been destroyed by the dmsdos +driver) and should be reported. If the command complains about errors but +the dos fs checker says everything is okay, you should also send a bug +report. + +To sync the write-back cache you can use 'dutil /mnt synccache'. +By default, the command waits until the data have been passed to the kernel's +filesystem buffering interface. (If you want to ensure that they are really +physically written issue a 'sync' afterwards which causes the kernel to +write all dirty buffers to disk.) You can specify an additional argument !=0 +in order to pass the cache contents to the daemon instead of writing directly +to disk (not useful if you want to ensure that the data are written *now*). + +Note that dutil displays two different compression ratios - an internal and +an external one. The internal one is always lower and is used to estimate +how much space is left on the device. It depends on which information can +actually be obtained from the MDFAT without scanning the whole disk (would +take too much time). If you want to compare different versions of compressed +filesystems to each other (e.g. doublespace vs. stacker) concerning the +compression effectivity, always use the external (second) one because it is +always calculated in the same manner. The external compression ratio is +calculated according to this formula: + + Space that would be used on an equivalent uncompressed msdos filesystem +------------------------------------------------------------------------- : 1 + Space that is used on the disk + +'Space' means what file and directory clusters occupy, it does *not* mean: +FAT, root directory, futher filesystem maintainance information. + + +10. The dmsdos daemon 'dmsdosd' +----------------------------------------------------------------------------- + +The dmsdos daemon is a program that runs in the background and +does delayed compression when the system is idle. This prevents the dmsdos +driver from slowing down your system when compressing large amounts of data. +In newer dmsdos versions, the daemon has also some influence on dmsdos +memory usage (it shrinks the amount of memory dmsdos eats for caching if +the dmsdos driver has not much to do). This might be an advantage for +systems that are short on memory. + +Actually, the daemon exists in two variants. The first one is a user-level +process (dmsdosd) that must be started with a command line +like 'dmsdosd /mnt' or 'dmsdosd /mnt xx', where /mnt is the mount point +and 'xx' is the compression level like in the cf:xx mount option. If the +xx is omitted it defaults to 12 (i.e. maximum compression). Please note +that this variant of the dmsdos daemon does not care about the dmsdos cf:xx +option or the dutil setcf command, but uses its own from its command line. +This variant of the dmsdos daemon is called 'external daemon'. For a summary +concerning this variant of the dmsdos daemon you can also take a look at the +dmsdosd manual page (file src/dmsdosd.1). + +The second variant is built inside the kernel and starts up automatically +when it is needed and exits when it is no longer needed. This variant is +called 'internal daemon' and cannot be killed from user space, not even by +root. It also obeys the cf:xx option and the dutil setcf command. + +You can only use one daemon at a time. You must decide before compiling +dmsdos which variant to use. During dmsdos configuration ('make config') +you are asked which one you want. Both daemon variants do the same work, but +the internal one is more user-friendly (well, you don't have to start it +manually :-) ) while the external one is more flexible and seems to eat up +less system time (but has other backdraws - you must kill it before you can +unmount the filesystem that has been given as a command line argument). If +you are using the internal daemon, the external one won't start - it always +exits with a "no permission" error. + +If you are going to use the external daemon, please note this: DON'T start +one daemon for each filesystem. The one single daemon does all the work for +the filesystem given in its command line and for all other dmsdos +filesystems you have currently mounted and for all other dmsdos filesystems +you mount later while the daemon is running. To prevent security problems +only root can start the external daemon (and even more, the driver refuses +to talk to a non-root dmsdos daemon via the ioctl interface). The external +daemon may be killed (by root) at any time, even when it is under hard work. +Send a SIGINT to it to let it finish its actual work and then exit. Send a +SIGTERM to it to make it exit immediately. In any case, the daemon catches +the signals and exits cleanly. + +The dmsdos daemon works like this: When it starts up, it tells the dmsdos +driver not to compress the data when doing write access to the compressed +filesystem. Instead, the driver writes the *uncompressed* data to the disk +and maintains a list of clusters that have to be compressed. When the daemon +has time to do something, it asks the driver for a cluster to compress. Then +the driver looks up a cluster in its 'clusters-that-must-be-compressed' +list, reads it from disk and passes it to the daemon. The daemon itself now +compresses the data. When it has finished compression, it passes the +compressed cluster back to the driver, which in turn writes it to the disk. + +There are two situations the list gets lost and the data that have not yet +been compressed by the daemon remain uncompressed on the disk: + + 1. Rebooting the system clears the whole list. + 2. Unmounting the filesystem clears all the list entries that + contain clusters of the filesystem to be unmounted. + +Please note that this kind of "cache" has a limited size. If you exceed that +size by writing *much* data at once the dmsdos driver no longer lets the +daemon do the job, but starts to compress the data itself. You will notice a +moderate system slowdown in that case. In extreme situations you might not +even see an immediate reaction on a keystroke, but those cases have become +rare. (If you suddenly see a *great* system slowdown this is usually caused +by too much fragmentation - not by compression.) + +The size of this "cache" can be selected during 'make config'. The number +specified there means the maximum (i.e. total, not per CVF) number of +clusters that can be in the list. If you want to know the amount in bytes, +multiply with the cluster size (which is 32KB for drivespace 3 and 8KB for +the other compressed filesystems). The dmsdos driver may use up to this +amount of _uncompressed_ disk space inside each compressed partition as +"cache". You can use dutil to find out how much uncompressed free space is +on the disk. If it is less than the "cache" size you shouldn't write more +than the amount of uncompressed free space to the disk at once. Otherwise, +the disk is getting full with the cached uncompressed data and you receive +a 'no space left on device' error. + +If there's nothing to compress any more the daemon goes to sleep mode, but +when the dmsdos driver needs the daemon it tries to awake it immediately. +Furthermore, the daemon awakes periodically to check for data and inform +the driver when it is time to release memory. + +Please note that using the daemon causes *duplicate* write access to the +filesystem. This results in a lot of more MDFAT level fragmentation. Thus, +the daemon cannot really make your system faster. In fact, it only makes +your system *appear* faster by doing a kind of load balancing. This load +balancing avoids unnecessary locks (kind of wait loops) in the driver. It +means the time some processes would have to wait can be used by other +processes. + +As I'm sometimes asked what to recommend: For a system with read-only or +very little write access and enough memory you don't need the daemon. For a +system with moderate write access and an appropriate fast processor the +internal daemon is best choice. For a system with moderate to high-traffic +write access to compressed partitions I recommend the external daemon. +If you have a slow processor and you need CPU time for some other intensive +computing, it's best to use the external daemon with a nice value (see +'man nice') - this prevents the daemon from slowing down the other +processes. If the daemon does not compress fast enough try to lower the +compression level in its command line. Well, and if you want to write a +large chunk of data (some 100MB) to a compressed partition at once, disable +the daemon before - it cannot help in this situation. It only causes too +much fragmentation which in turn slows down the system horribly. + +Just another hint: If you want to use the daemon only for memory management +but not for delayed compression, you can clear speedup bit #7. + +You see, there's much room for experimentation here, especially if you use +the external daemon :) There are some pre-configured dmsdos config files +which just implement some of the above recommendations. This is explaind +in INSTALL.TXT so see there for details, please. + + +11. The dmsdos library 'libdmsdos.a' +----------------------------------------------------------------------------- + +This feature is currently experimental, but begins to become more and more +interesting. + +The dmsdos library is intended to provide some low-level dmsdos functions +to user processes. The library is not new source code - it's just the +dmsdos module source compiled in a different way (well, with some tricks). +The dmsdos library aims towards working with raw CVFs, not filesystems. +It is usable only for *not* mounted CVFs. Well, for mounted ones there's +the dmsdos module :) + +Documentation (including a detailed description of the interface functions) +can be found in file libdmsdos.doc. + + +12. The dmsdos filesystem checker 'dmsdosfsck' +------------------------------------------------------------------------------ + +The goal of dmsdosfsck is not having a complete and reliable filesystem +checker for CVFs. If you want that, use the filesystem maintainance tools +that came with your CVF software package under Dos (or blame their +manufacturers if they are not working well...). dmsdosfsck is just meant +as a kind of workaround for problems that may show up if the dmsdos or +FAT driver suddenly detects an unexpected inconsistency and might get +confused. With dmsdosfsck those problems that may confuse the drivers can +be detected and corrected before mounting the CVF. + +The command line of dmsdosfsck looks like this: + +dmsdosfsck [-a] [-r] [-v] [-l] device|file + +where -a means repair automatically (uh... be VERY careful), -r means +repair interactively, -v means be verbose, and -l means list the files +while processing them. These options are somewhat similar to the well-known +dosfsck. + +Currently not all errors can be corrected. It just corrects fatal FAT +errors like loops, crosslinks, illegal values, and unexpected ends of +chains - things the kernel FAT driver does not like at all and tends to +crash or hang if it runs over them. (Well, this is a question of philosophy. +Should a kernel driver be idiot proof or may it assume the filesystem is +valid in most cases at the most vital places? I don't want to give an +answer, but the FAT driver of Linux 2.0.34 seems not 100% idiot proof.) +So there is really a need for a dmsdosfsck utility here. + +Furthermore, it can detect but not correct MDFAT errors, crosslinks, +damaged MDFAT entries - you must repair them under Dos using the CVF +maintainance tools that came with your CVF software package. The kernel +FAT driver does not see these structures, and the dmsdos driver simply +returns read errors - those errors just cannot confuse drivers or even +make the system unusable. + +It can detect and correct BITFAT allocation errors - this is necessary +for write access. (But the dmsdos module already does it if it is told +to do so at mount time.) + +It detects damaged directory entries and can correct them. This is also a +situation the kernel FAT driver doesn't like at all. Be warned, repairing +damaged directories is a very critical job which cannot be automated +completely. Some damaged directories cannot be repaired and will be converted +into files instead. This gives you a chance to take a low-level disk editor +and repair this by hand under Dos later (if you know how to do it). Repaired +directories are saved without compression so you should be able to analyse +them under Dos without further help. + +It also checks whether one filename is used more than once in a directory. +The file can optionally be renamed. This is usually done by appending ~%d +to the filename where %d is the lowest possible number not causing a +conflict. + +It also detects lost cluster chains but does not correct them. They just +use space and do nothing evil. For that, use the Dos tools that came....:) + + + *** Currently, dmsdosfsck must be considered extremely alpha test *** + + +It's far away from complete, but some useful things work. You *can* +use dmsdosfsck with the generic fsck frontend. You can even +run your usual fsck when mounting from /etc/fstab at boot time, even on +CVF partitions. But that's not trivial. If you really want to do such +things, take a look at the example script 'msdos.fsck-wrapper' which is +suitable for the generic fsck frontend. Be sure to understand what the +script does. Change it to your requirements. Then replace the link in +/sbin/fsck.msdos (or /etc/fs/fsck.msdos) by a link to the script :) + + +13. The dmsdos Midnight Commander external filesystem utility 'mcdmsdos' +------------------------------------------------------------------------------ + +Dmsdos comes with an interface utility that accepts standard Midnight +Commander commands for reading archive files. The utility is named +'mcdmsdos' and is compiled during usual dmsdos compile process. +The utility is currently read-only. + +Please refer to the documentation of Midnight Commander for further +information on how to configure an external filesystem. +You may want to write a small shell script as a wrapper around mcdmsdos +in order to suppress output to stderr distorting the screen, e.g. + + #!/bin/sh + exec /usr/local/bin/mcdmsdos $@ 2> /dev/null + +Furthermore, mcdmsdos can be used as a utility to quickly look at what is +inside the CVF and to extract single files from a CVF. Today mcdmsdos even +works in a Win32 environment, so it may be worth to learn its command line +parameters :)) + + mcdmsdos list <CVF_name> + + lists (recursively) the contents of the CVF, i.e. the names of all + files that are in that compressed partition are printed. + + mcdmsdos copyout <CVF_name> <path/name_of_file> <outfile> + + extracts the file <path/name_of_file> from the CVF <CVF_name> and + writes it to <outfile> (which must be a real file, not stdout). + + mcdmsdos copyin <CVF_name> <path/name_of_file> <infile> + + is expected to copy <infile> into the CVF <CVF_name> at + <path/name_of_file> according to Midnight Commander documentation. + This command is currently not implemented as the utility is + read-only. + +If documentation of newer versions of Midnight Commander wants some more +commands consider the respective operations to fail. I expect something +like remove, rmdir, mkdir etc. which are missing in the current standard +(I wrote this utility according to Midnight Commander 3.0 documentation). +But that doesn't matter as long as mcdmsdos is read-only :) + + +14. Authors and email addresses: +------------------------------------------------------------------------------ + +CVF-FAT/dmsdos is the official successor of the former dmsdos filesystem. The +dmsdos filesystem was initially written from scratch by Frank Gockel, after +taking a close look at the THS filesystem (a read-only doublespace fs with +only DS-0-2 decompression written by Thomas Scheuermann). Stacker support +and drivespace 3 'ultra' compression/decompression were added by Pavel Pisa +as well as improved DS/JM compression and decompression routines. Meanwhile, +it contains several parts of code that was directly provided or code that is +based on the ideas from a lot of people over the net in order to fix bugs, +to improve performance, and to add features. + +The dmsdos code is distributed under the GNU General Public Licence +(see file COPYING). + +The dmsdos driver is currently maintained by Pavel Pisa (stacker access, +SQ compression, meanwhile all other compression too) and me, Frank Gockel +(rest of the code). + +Pavel's email address is [email protected], my email address is + +If you want to contact me via email, please write in English or take a close +look at the country codes in the email addresses :) diff --git a/doc/ioctl.doc b/doc/ioctl.doc new file mode 100644 index 0000000..77c9f96 --- /dev/null +++ b/doc/ioctl.doc @@ -0,0 +1,199 @@ +DMSDOS ioctl commands (outdated.... newers are missing here) + +***************************************************************************** +WARNING: INFORMATION IN THIS FILE IS OUTDATED AND PARTIALLY WRONG. SEE THE + DMSDOS UTILITIES SOURCE CODE FOR PROGRAMMING EXAMPLES. + + When I have enough time I write better documentation..... +***************************************************************************** + +Example code: + + #include<stdio.h> + #include<linux/dmsdos_fs.h> + #include<sys/ioctl.h> + #include<sys/types.h> + #include<sys/stat.h> + #include<fcntl.h> + #include<string.h> + #include<errno.h> + + Dblsb dblsb; + int fd; + int ret; + unsigned long w[10]; + Mdfat_entry mde; + + /* open file descriptor fd */ + fd=open(CVF_name,O_RDONLY); + if(fd<0) + { printf("No such file or directory.\n"); + return; + } + +DMSDOS_GET_DBLSB: +read DMSDOS version number and read CVF parameters in dblsb structure. +This function also performs a version check, i.e. if the dmsdos driver in +the kernel thinks the version number you supply in s_dcluster is too old, +it "or"s the returned value with 0x0f000000 and doesn't fill in the values +for the dblsb structure. + +** Since not all DMSDOS versions support all commands, this ioctl should be +always issued first ** + + dblsb.s_dcluster=DMSDOS_VERSION; + ret=ioctl(fd,DMSDOS_GET_DBLSB,&dblsb); + if(ret<0) + { printf("This is not a DMSDOS directory.\n"); + exit(1); + } + printf("You are running DMSDOS Version %d.%d.%d.\n\n",(ret&0xff0000)>>16, + (ret&0x00ff00)>>8,ret&0xff); + if(ret&0x0f000000) + { printf("This program is too old for the DMSDOS version you are running.\n"); + exit(1); + } + + dblsb structure see dmsdos.h. + +DMSDOS_READ_BITFAT: +read bitfat entry +w[0]=sectornr; returns 0 in w[1] if sector free + + ret=ioctl(fd,DMSDOS_READ_BITFAT,w); + if(ret<0)error.....; + +DMSDOS_WRITE_BITFAT: +write bitfat entry +w[0]=sectornr, w[1]=new value (0=free, 1=allocated) + + ret=ioctl(fd,DMSDOS_WRITE_BITFAT,w); + if(ret<0)error.....; + +DMSDOS_READ_MDFAT: +read mdfat entry +w[0]=clusternr; returns result in mde structure. + + w[1]=&mde; + ret=ioctl(fd,DMSDOS_READ_MDFAT,w); + if(ret<0)error......; + +DMSDOS_WRITE_MDFAT: +write mdfat entry +w[0]=clusternr; mde=new mdfat entry value. + + w[1]=&mde; + ret=ioctl(fd,DMSDOS_WRITE_MDFAT,w); + if(ret<0)error......; + +raw MDFAT entry: 32/40 bit number (hidden to the user) + + dos 6.x, Win95 without drivespace 3: + 3322222222221111111111 + 10987654321098765432109876543210 + ucaaaammmm?sssssssssssssssssssss + + Win95 with drivespace 3: + 33333333322222222221111111111 + 9876543210987654321098765432109876543210 + ucaaaaaammmmmm??ssssssssssssssssssssssss + +Mdfat_entry structure: (the dmsdos driver automatically converts it to the + appropriate 32 or 40 bit number) + + mde.sector_minus_1= ('ssss...' bits) + mde.size_lo_minus_1= ('mmmmmm' bits) + mde.size_hi_minus_1= ('aaaaaa' bits) + mde.flags= ('uc' bits) + mde.unknown= ('?' bit(s)), are NOT written, always set to zero + + u: 1=entry is used, 0=entry is free + c: 1=cluster is uncompressed, 0=cluster is compressed + a: uncompressed size minus 1 (number of sectors belonging to this cluster) + m: compressed size minus 1 ( " " " ) + ?: unknown, but seem(s) to be used sometimes + s: sector number minus 1 (theoretical 2 GB limit) + The sector number is counted from the beginning of the CVF file + starting with sector number 0. + +DMSDOS_SET_COMP: +set compression method for write access. + + ioctl(fd,DMSDOS_SET_COMP,READ_ONLY); or: + ioctl(fd,DMSDOS_SET_COMP,UNCOMPRESSED); or: + ioctl(fd,DMSDOS_SET_COMP,GUESS); or: + ioctl(fd,DMSDOS_SET_COMP,DS_0_0); or: + ioctl(fd,DMSDOS_SET_COMP,DS_0_1); or: + ioctl(fd,DMSDOS_SET_COMP,DS_0_2); or: + ioctl(fd,DMSDOS_SET_COMP,JM_0_0); or: + ioctl(fd,DMSDOS_SET_COMP,JM_0_1); or: + ioctl(fd,DMSDOS_SET_COMP,SQ_0_0); + +DMSDOS_SET_CF: +set compression level: + + ioctl(fd,DMSDOS_SET_CF,level-1); + +DMSDOS_EXTRA_STATFS: +special statfs command for dmsdos +reads special info in w. + + ioctl(fd,DMSDOS_EXTRA_STATFS,w); + +w[0]= free sectors in CVF. +w[1]= used sectors. +w[2]= maximum free unfragmented sectors (contiguous sectors) +w[3]= free fat clusters +w[4]= used fat clusters +w[5]= clusters allocated in fat but free in MDFAT (filesystem error if > 0) +w[6]= sum of sectors used by compressed clusters +w[7]= sum of sectors used by uncompressed clusters +w[8]= number of compressed clusters +w[9]= number of uncompressed clusters + +DMSDOS_READ_BLOCK: +read sector from CVF; returns sector contents in bstruct.data; + + struct bstruct + { unsigned long sectornr; + unsigned char data[512]; + }; + + bstruct.sectornr= sector number; + + ioctl(fd,DMSDOS_READ_BLOCK,&bstruct); + +DMSDOS_WRITE_BLOCK: + + analogue to read_block... + +DMSDOS_READ_DIRENTRY: +DMSDOS_WRITE_DIRENTRY: + + removed and unsupported... + +DMSDOS_READ_DFAT: +w[0]=clusternr; returns fat entry value in w[1]; (last=-1 or 0xFFFFFFFF) + + ret=ioctl(fd,DMSDOS_READ_DFAT,w); + if(ret<0)error......; + +DMSDOS_WRITE_DFAT: +w[0]=clusternr; w[1]=new value; + + ret=ioctl(fd,DMSDOS_WRITE_DFAT,w); + if(ret<0)error......; + +DMSDOS_SIMPLE_CHECK: +performs simple filesystem check (crosslinks & allocation errors) +w[0]=0; means do not try to repair errors + +returns result in w[0] (0=ok, -1=FAT error, -2=MDFAT error, -3=BITFAT error, + 1=out of memory, check aborted, + 2=FAT ok, but out of memory for MDFAT check) + + ioctl(fd,DMSDOS_SIMPLE_CHECK,w); + if(w[0]==....)printf(.....); + + +dmsdos daemon ioctls missing here.... diff --git a/doc/libdmsdos.doc b/doc/libdmsdos.doc new file mode 100644 index 0000000..8d4ce70 --- /dev/null +++ b/doc/libdmsdos.doc @@ -0,0 +1,614 @@ +This is documentation for the dmsdos library. 10Sep1998 + + +The dmsdos library, libdmsdos.a, provides the dmsdos interface that is known +from the kernel to a user-level process. This is acchieved by compiling the +dmsdos code simply in a different way (well, with some rather dirty tricks). + +The library is intended to export the following functions: + + open_cvf + close_cvf + dbl_bitfat_value + dbl_fat_nextcluster + dbl_mdfat_cluster2sector + dbl_mdfat_value + dmsdos_read_cluster + dmsdos_write_cluster + raw_bread + raw_brelse + raw_getblk + raw_mark_buffer_dirty + raw_set_uptodate + simple_check + stac_bitfat_state + +In fact, it exports much more, but only these listed here are guaranteed +to be kept in future versions. Well, one thing should be important to know: +The library does NOT export virtual sector management. It will never do. + +The functions are just used like in the kernel. For prototypes, see file +dmsdos.h. How to use them see the sources. There's some example code +how to use the library (dcread.c). + +The first two, open_cvf and close_cvf, are special. They are called instead +of mounting and unmounting the CVF. + +How to "mount" a CVF (example code): + + #include "lib_interface.h" + #include "dmsdos.h" + + struct super_block*sb; + sb=open_cvf("CVF_Filename",rwflag /*0=RDONLY or 1=RDWR*/); + if(sb==NULL) + { fprintf(stderr,"open CVF failed\n"); + exit(1); + } + +Keep the sb pointer. It must be passed to a lot of functions. It can be used +in the same way as in the kernel. The sb pointer is used to distinguish +between several open CVFs (in fact, in sb->s_dev the file descriptor is +stored). + +After use, the CVF must be "unmounted" again: + + close_cvf(sb); + +If you want to see more example code, look at the dcread utility source +(file dcread.c in the src directory). + +Notes: + * The user-level functions are single-threaded. You cannot run several + processes on the same CVF unless all are only reading. The library + uses flock in order to detect such situations and just denies access + in that case. You can however use different CVFs in parallel, even + read-write, even in one program. + * You should use this library only to access a CVF that is currently + _not_ mounted. There's a high risk of destroying the CVF or even + crashing the system otherwise. + * You should not touch CVFs that are currently used by dosemu with + wholedisk or partition access. Again there's a high risk of destroying + the CVF or crashing dosemu otherwise. + * If you do not intend to write to the CVF, open it in read-only mode. + Then the CVF is not locked exclusively and other processes are allowed + to read the CVF at the same time. + * The library prints the "kernel" messages to stderr. It obeys the + loglevel variable. The only way to make libdmsdos quiet is to + direct stderr into a file or to /dev/null. + * The first call of open_cvf initializes the library and prints some + version information to stderr. + * open_cvf does not do a filesystem check. If you do want the same + simple filesystem check that dmsdos usually does when mounting, call + simple_check afterwards. + * open_cvf may open the filesystem read-only though you request + read-write access. This happens if write access is denied or if the + dmsdos subsystem detects unexpected problems with the CVF. You may + not like this behaviour, but it's best for long life of your CVFs :) + Programmers please check sb->s_flags for the MS_RDONLY flag after + calling open_cvf to be sure. + * To compile your application using the dmsdos library you should under + no cirumstances define the macro __KERNEL__ for the whole program. + It causes problems with libc6. + +As an idea what the dmsdos library might be good for: + + * A mtools clone for CVFs, for example called dtools. (A simple program, + dcread, exists but its quatilty cannot be compared to mtools. Sorry.) + * A fsck.dmsdos. (An incomplete alpha test version exists.) + * A defragmentation program. (Not yet written.) + * For debugging dmsdos at user level (it's the same source code). + * An external fs for midnight commander. (A read-only interface program, + mcdmsdos, exists.) + * ... add your favourite idea here :) + * ... I've even received mail about whether it would be possible to + port libdmsdos to Win32 and use it for a Win95/98/NT driver :)) + +Support for a shared dmsdos library +----------------------------------- + +libdmsdos can be compiled as a shared library. You must edit the +Makefile in the src directory for that purpose. Note that the default +installation does not compile the shared library. This is intended. If +you are not an experienced shared library hacker please be careful. It's +easy to screw up some binaries with shared libraries :) + +You should not use the dmsdos shared library for now as there's currently no +standard that ensures that future versions will be backwards compatible. +Link statically with libdmsdos.a unless disk space or memory is very critical. + +WARNING: If there's a shared dmsdos library lying around, gcc defaults to +linking *dynamically* all programs that need this library. For example, if +libdmsdos.so is present and you recompile dmsdosfsck, the binary will be +dynamically linked against that library. This can be very confusing and may +not do what you want :( + +Built-in direct translation access mode +--------------------------------------- + +This feature has been added to the library for convenience. It only works +for FAT12 or FAT16 MSDOS filesystems. It does not work, for example, for +CVFs on a cdrom or in a FAT32 or non-FAT filesystem. + +The dmsdos library can access a CVF that resides in an uncompressed FAT12 or +FAT16 MSDOS host partition even if the uncompressed host partition is not +mounted (this was programmed in fact by cut'n'paste of some old dmsdosfs +functions). If there are more than one CVF in that partition, the first +valid CVF found in the root directory of the uncompressed host partition +is used. If you want to select one of more CVFs, append a colon and the +extension of the CVF to the filename. + +For example, if your uncompressed host partition is /dev/hda1 and you want +to access the CVF DRVSPACE.001 on that partition, use "/dev/hda1:001" as +filename when calling open_cvf. If DRVSPACE.001 is the only CVF inside +/dev/hda1 you can even use "/dev/hda1" and it will find the CVF. + +This special feature allows, for example, dmsdosfsck to check a CVF in +a partition that has not yet been mounted. This may be useful at boot time +to check CVFs :) + +Standard C library functions that libdmsdos needs +------------------------------------------------- + +If you want to use libdmsdos in an environment where the standard C library +functions are not available then you must write an emulation for all these +functions and link your code against it instead of the standard C library. +The emulation needn't be full-featured like the C library. Look at the source +if you are in doubt. + + close (4) + errno + exit (3) + flock (only if compiled with -DUSE_FLOCK) (4) + sopen (only if compiled with -DUSE_SOPEN) (4) (8) + fprintf (2) + free (6) + lseek (1) (7) + malloc (6) + memcpy + memmove + memset + open (only if _not_ compiled with -DUSE_SOPEN) (8) + read (1) + strcat + strerror + strncat + strncmp + strncpy + strrchr + time (5) + vsprintf (2) (9) + write (1) + + (1) only used in aligned 512 byte blocks + (2) only used for logging to stderr, no other files are used + (3) used for abort in case of fatal error + (4) locking may be left out + (5) a time-proportional counter would be enough, needn't be exact + (6) called quite often, be aware of possible memory fragmentation + (7) only used on 512 byte boundaries with SEEK_SET, also used to find out + file size with SEEK_END + (8) must parse filename (invent your own syntax if necessary) + (9) a free, portable emulation is in linux/lib/vsprint.c + +If possible, please avoid hacking around in dmsdos kernel code if the library +does not work in your environment (unless it's a bug). The only files that +should be modified due to OS or compiler differences are lib_interface.c and +lib_interface.h. Please surround your changes with appropriate #ifdef's so +the modified code still compiles under all those environments where it worked +before. Please do not violate portability of the code. Hardware comes and +goes, OSses are born and die, but portable code survives :) + + +dmsdos library reference guide +------------------------------ + +All functions need the header files lib_interface.h and dmsdos.h. Error +checking is not always possible. This is because some of the low-level kernel +functions dmsdos calls when running as kernel module never fail. In user +space, there may be failures, however. This means, libdmsdos is currently +not 100% error-safe. As a workaround (that works under the standard C library) +you can set errno to NO_ERROR before calling the function and afterwards +check whether it has changed to an error value. + + open_cvf: + + struct super_block* open_cvf(char*filename,int rwflag) + + Opens CVF described by filename. rwflag is 0 for read-only, otherwise + the file is opened read-write. The file is locked read-only or + read-write, depending on rwflag (by calling flock or sopen). + + filename is interpreted in a special way if it contains a colon. + See chapter 'Built-in direct translation access mode' above. The colon + and the rest of the string after it are stripped off before the filename + is forwarded to the libc function open or sopen. + + Return Value: Pointer to valid super_block structure (which can be + interpreted as a kind of file handle), NULL if failed. + + Note: Keep the super_block pointer. Most dmsdos functions need it in + order to identify the CVF. + + Note: The CVF may be opened read-only though you opened it in read-write + mode. This happens if the dmsdos code detects unexpected errors in the + CVF. Please check sb->s_flags for the MS_RDONLY bit after calling + open_cvf. + + close_cvf: + + void close_cvf(struct super_block*sb) + + Closes the CVF. If necessary, the file is unlocked before. + + dbl_bitfat_value: + + int dbl_bitfat_value(struct super_block*sb,int sectornr,int*new) + + Reads or writes bitfat value that belongs to sector sectornr. + *** This is a low-level function you are probably never interested in. + Bitfat values are CVF format specific. You do not need to know them + unless debugging write access :)) + + If new is NULL the value is read and returned. + Otherwise the actual value is replaced with that one found in *new. + + Return value: read access: read value. write access: undefined. A + negative value indicates an error. + + dbl_fat_nextcluster: + + int dbl_fat_nextcluster(struct super_block*sb,int clusternr,int*new) + + Reads or writes the FAT entry for cluster clusternr. (You should know + what a FAT is, otherwise please read a good book on Dos disk access.) + + If new is NULL the value is read and returned. + Otherwise the actual value is replaced with that one found in *new. + + Return value: read access: read value. write access: undefined. + -1 means the cluster is marked as last cluster in a file or an error + occured. Errors may also be represented by other negative results. + + Note: -1 can also be used for write access in order to indicate end of + file mark. It is automatically converted to the right value for the FAT + size. + + Note: if clusternr is invalid a -1 is returned. This usually makes + runaway FAT reading loops break as if there was an EOF in the cluster + chain. There's no means to find out whether a -1 is an error or an EOF + except making sure a valid cluster number is given. + + dbl_mdfat_cluster2sector: + + int dbl_mdfat_cluster2sector(struct super_block*sb,int clusternr) + + This is a low-level function that reads the mdfat and extracts the + starting sector information from it. It may be useful for write + access with dmsdos_write_cluster. It returns the starting sector number + of cluster clusternr. + *** Do not use the standard formula that is used in a normal FAT + filesystem (multiplying the cluster number with the number of sectors + per cluster and adding some offsets) - that formula is not valid in a + compressed filesystem. Use dbl_mdfat_cluster2sector instead. + + Return value: starting sector of the cluster clusternr. If negative, + an error occured. + + Note: see dmsdos_write_cluster what this function is useful for. + + dbl_mdfat_value: + + int dbl_mdfat_value(struct super_block* sb,int clusternr, + Mdfat_entry*new,Mdfat_entry*mde) + + Reads or writes the mdfat/allocmap entry that belongs to cluster + clusternr. + *** This is a low-level function you are probably never interested in. + Mdfat values are CVF format specific. You do not need to know them + unless debugging dmsdos :)) + + If new is NULL the value is read and stored in *mde. + Otherwise the actual value is replaced with that one found in *new. + + Return value: A negative value indicates an error. + Note: consider *mde as undefined after write access. + + dmsdos_read_cluster: + + int dmsdos_read_cluster(struct super_block*sb, + unsigned char*clusterd, int clusternr) + + Reads cluster clusternr in clusterd. The dmsdos library handles all + low-level access including decompression of the data. + + *** Be sure to have reserved enough memory for the data that this + function writes into clusterd. Usually, use a full cluster size. + How to determine the full cluster size see the dcread example code. + + Return value: number of bytes actually read (clusters may be shorter + than the full size in a CVF, which is not possible in an uncompressed + FAT filesystem). Usually, you can ignore this unless you use write + access. (A cluster may even have length zero.) Just in case, if the + cluster is shorter than full size the unused slack is zero'd out. On + error, a negative value is returned. + + Note: This function cannot read the root directory (cluster 0). You must + use low-level disk access for that (i.e. raw_bread). See also the dcread + example code. + + dmsdos_write_cluster: + + int dmsdos_write_cluster(struct super_block*sb, unsigned char* clusterd, + int length, int clusternr, int near_sector, + int ucflag) + + Writes cluster clusternr from clusterd back to disk. The dmsdos + library handles all low-level disk access including compression. + + length is the position of the last valid byte plus 1 in clusterd. + This is usually the value that dmsdos_read_cluster returned when you + read this cluster before unless you modified some bytes beyond that + value. The idiot-proof value is always the full cluster size. + *** Warning: if length is too low, the data may be truncated during + write access. If it is too high disk space is wasted. + + ucflag is a flag that indicates whether libdmsdos should try to + compress the data. 0 means try to compress, 1 means write the data + without compression. Please do not use other values than 0 and 1 + (they do exist but have some very special meanings). + + near_sector can be used to control where libdmsdos tries to place + the data on disk. For good performance on large files, it should + be set to the start sector value of the cluster that preceeds the + actual cluster in the file. The start sector value can be obtained with + dbl_mdfat_cluster2sector on that preceeding cluster. (This way actually + helps avoiding low-level fragmentation _and_ file data fragmentation in + the CVF.) + *** Do not use the standard formula that is used in a normal FAT + filesystem (multiplying the cluster number with the number of sectors + per cluster and adding some offsets) - that formula is not valid in a + compressed filesystem. + + The near_sector value is just meant as a hint. If there's not enough + free space on the disk around near_sector, dmsdos writes the data + somewhere else. If near_sector is set to zero, dmsdos uses an internal + way to find out a good value that avoids low-level fragmentation but + that is not necessarily good for performance on the file that is being + written. Nevertheless, 0 always works. + + WARNING: If you are writing a directory cluster you should not compress + the data (it would hurt performance a lot). Some of the original CVF Dos + software versions never compress directories, so you shouldn't do + either (it may cause compatibility problems otherwise). You also should + always use full cluster size for directories on some CVF versions. In + order to feel safe you can use the DIR_MAY_BE_COMPRESSED and + DIR_MAY_BE_SHORT macros. Have a look at their definition in file + dmsdos.h. + + Return value: A negative value indicates an error. + + Note: The data in clusterd are not destroyed by this function. + + raw_bread: + + struct buffer_head* raw_bread(struct super_block*sb,int block) + + Read a 512 byte block from the CVF. The data are read without further + modification. Block numbering scheme begins with zero. + + Return value: pointer to a buffer_head structure that contains the 512 + bytes of data that represent the data block. The data are in + buffer_head->b_data in an unsigned character array (see the dcread + example). On error, NULL is returned. + + Note: The data are returned in dynamically allocated memory. If you + do not need the data any more, you must free them by calling raw_brelse + in the buffer_head structure pointer. + + Note: You may modify the data. You must call raw_mark_buffer_dirty + after modification (this writes the data back to disk) and before + raw_brelse. + + Note: If you want to overwrite the whole block, use raw_getblk instead + of raw_bread (this leaves out the unnecessary disk read access). + + raw_brelse: + + void raw_brelse(struct super_block*sb,struct buffer_head*bh) + + Release dynamic memory for the data read by raw_bread. + + raw_getblk: + + struct buffer_head* raw_getblk(struct super_block*sb,int block) + + Like raw_bread, but leaves out the actual disk access. This is intended + to leave out unnecessary disk read access if you want to overwrite the + whole block. Consider the data as undefined. See raw_bread. + + Note: raw_getblk should be followed by a call of raw_set_uptodate + for compatibility. See there. + + raw_mark_buffer_dirty: + + void raw_mark_buffer_dirty(struct super_block*sb,struct buffer_head*bh, + int dirty_val) + + Write data back to disk after they have been modified. + + dirty_val must be always 1 (due to a misunderstanding/bug in dmsdos). + + Note: This function does not indicate an error. This is bad, but it is + because the Linux kernel interface also does not return error codes with + this function. As a hack (that works with libc) you can set errno to + NO_ERROR and check it afterwards, but note that this will not work in + the dmsdos kernel driver. Usually, there cannot be an error during that + kind of write access unless the disk has bad sectors. + + raw_set_uptodate: + + void raw_set_uptodate(struct super_block*sb,struct buffer_head*bh,int v) + + No longer used (does nothing in libdmsdos). + + For compatibility with the kernel driver, this should be called with + v=1 after raw_getblk on the buffer_head pointer that raw_getblk returned + (unless it's NULL due to failure). + + simple_check: + + int simple_check(struct super_block*sb,int repair) + + Check some internal filesystem tables for consistency. + + If repair is 1 libdmsdos tries to correct errors (the CVF must have been + opened read-write in that case). If it is 0 the tables are just checked. + + Return value: a negative value indicates an error, i.e. either the + tables are inconsistent or the errors, if any, could not be repaired. + + Note: It is strongly recommended to run simple_check immediately after + opening the CVF if you want to write to it. Do not write to a CVF if + simple_check fails on it (you may detroy some files in the CVF). As + CVFs are known to be very prone to so-called bitfat errors, you probably + want to give the program that you are writing a kind of "expert option" + that lets simple_check run with repair=1. The same warnings apply as to + the dmsdos mount option bitfaterrs:repair :)) + + stac_bitfat_state: + + int stac_bitfat_state(struct super_block*sb,int new_state) + + Stacker has a kind of clean/dirty flag that indicates whether the CVF + was correctly unmounted. This function can be used to manipulate it. + *** This is a low-level function you should not use unless you are a + dmsdos and Stacker expert. See the dmsdos source code. + + Note: Do not call this function on non-Stacker CVFs. + + +Programming notes +----------------- + +In a normal libdmsdos application, you open the CVF with open_cvf. If you +intend to write to the CVF later, you should call simple_check after that +and refuse write access if an error is found. + +Then you use dmsdos_read_cluster and dmsdos_write_cluster to access the +CVF. Only for the root directory you need raw_bread, raw_mark_buffer_dirty +and raw_brelse. For following files you need to read the FAT with +dbl_fat_nextcluster. + +You write to the FAT with dbl_fat_nextcluster in order to truncate files or +append data to them, or even to delete files and directories and to create +new ones - all like in a usual FAT filesystem. + +If you are done, don't forget to close the CVF with close_cvf. + +Here is a simple example that illustrates the usage of all important +libdmsdos functions. You should not compile and execute this example as +it's not always correct C syntax. Things like error handling have been left +out to simplify the code (which is not good). Furthermore, the program just +pokes around in the CVF which would be bad for your data :) + + + #include "lib_interface.h" + #include "dmsdos.h" + + struct super_block*sb; + struct Dblsb*dblsb; + + /* open the CVF read-write */ + sb=open_cvf("dblspace.001",1); + if(sb==NULL)...error... + + /* check the CVF */ + if(simple_check(sb,0)<0)...error... + + /* read and display cluster 2 as hex numbers */ + int i; + int size; + int full_size; + unsigned char* clusterd; + + /* determine full size */ + dblsb=MSDOS_SB(sb)->private_data; + full_size=dblsp->s_sectperclust*512; + + clusterd=malloc(full_size); + if(clusterd==NULL)...error... + + size=dmsdos_read_cluster(sb,clusterd,2); + if(size<0)...error... + for(i=0;i<size;++i) printf("%x ",clusterd[i]); + printf("\n"); + + /* modify one byte at position 4234 and write cluster 2 back */ + int pos; + + pos=4234; + clusterd[4234]=123; + if(size<=pos)size=pos+1; /* adapt size */ + + if(dmsdos_write_cluster(sb,clusterd,size,2,0,0)<0)...error... + + free(clusterd); /* free memory that is no longer needed */ + + /* determine which cluster follows cluster 2 in fat */ + int fatval; + + fatval=dbl_fat_nextcluster(sb,2,NULL); + if(fatval==0) printf("cluster 2 is unused\n"); + else if(fatval==-1) printf("cluster 2 is marked as EOF\n"); + else printf("cluster 2 is followed by cluster %d in FAT\n"); + + /* mark it as EOF */ + int new; + + new=-1; + dbl_fat_nextcluster(sb,2,&new); + + /* read root directory */ + int i; + unsigned char*data; + int number; + struct buffer_head*bh; + + dblsb=MSDOS_SB(sb)->private_data; + number=dblsb->s_rootdirentries*32/512; + startblock=dblsb->s_rootdir; + + data=malloc(number*512); + if(data==NULL)...error... + + for(i=0;i<number;++i) + { bh=raw_bread(sb,startblock+i); + if(bh==NULL)...error... + memcpy(data+i*512,bh->b_data,512); + raw_brelse(sb,bh); + } + /* complete root directory is now in data */ + + /* modify root directory */ + ...modify data where you like... + + /* write it back completely */ + for(i=0;i<number;++i) + { bh=raw_getblk(sb,startblock+i); /* or raw_bread(...) */ + if(bh==NULL)...error... + raw_set_uptodate(sb,bh,1); /* leave out if raw_bread used above */ + memcpy(bh->data,data+i*512,512); + raw_mark_buffer_dirty(sb,bh,1); + raw_brelse(sb,bh); + } + + free(data); /* free memory that is no longer needed */ + + /* we are done */ + close_cvf(sb); + + +For a piece of code that actually compiles and runs and does something +eventually useful and ... does not kill your data :-)) see the dcread +example program (file dcread.c). diff --git a/doc/messages.doc b/doc/messages.doc new file mode 100644 index 0000000..71c6294 --- /dev/null +++ b/doc/messages.doc @@ -0,0 +1,871 @@ +dmsdos driver: kernel messages in alphabetical order. + +*** Some messages need a better explanation. Under construction :) *** + +Classes: I: Information. + E: Error in filesystem, check it under dos (incl. the surface test). + O: Other problem, not a filesystem error. + B: Bug, please send a bug report. + P: Panic message. Extremely severe filesystem, hardware or dmsdos + error. The process that actually called dmsdos when the problem + occured has been "frozen" in order to prevent data corruption. + Though your system continues to run, you should reboot immediately + and check the compressed filesystems. You may have to press the + reset button in order to reboot. + ?: Unknown. (Look at the source code.) + +------------------------------------------------------------------------------ + +DMSDOS: adapting free sectors count + I: Stacker sector count is corrected. + +DMSDOS: BB_ClustCnt=0x%x impossible (FAT32?) + E/O?: The filesystem is in an unsupported format, damaged or otherwise + strange. Dmsdos refuses to mount it. Please verify whether it is FAT32 + under Dos/Win95. Currently compressed FAT32 is not supported. + (Compressed FAT32 has never been reported to exist so far.) + +DMSDOS: bitfat cache: + I: Debug message. Ignore. + +DMSDOS: BITFAT mismatches MDFAT, repairing... +DMSDOS: BITFAT mismatches MDFAT (sector %d is %d and should be %d) +DMSDOS: BITFAT mismatches MDFAT (sector %d) + E: There's an error in the allocation table (BITFAT) of your compressed + partition - next time you boot dos you should check it. WARNING: + Writing to an errorneous filesystem may cause further damage. + Optionally (with mount option bitfaterrs=repair) + the driver tries to repair the problem by recalculating the BITFAT. + +DMSDOS: bmap called, unsupported! + B: Shouldn't happen. + +DMSDOS: cannot write dirty cluster %d on dev 0x%x, trying again later + E/O: Very likely the filesystem is full. There should be another message + describing more details. The driver keeps the cluster in memory. This + is a very critical condition as you may lose the data in that cluster. + The driver usually tries 5 times to write the data again before + throwing them away and setting the filesystem to read-only mode. If + the filesystem is really full you still have a chance to free up some + space now. + +DMSDOS: ccache contents: +DMSDOS: ccache statistics: + I: Debug messages. Ignore. + +DMSDOS: ch_dirty(_locked): READ-ONLY filesystem + O: When the filesystem suddenly is set read-only (e.g. because of a + problem) some kernel functions seem not to obey the RO flag. In this + case, the dmsdos driver itself rejects the command in order to + avoid trashing the filesystem. + +DMSDOS: ch_read: no memory! + O: System memory is even too low for read access. See below for hints + about 'no memory' messages. + +DMSDOS: ch_read: read_cluster failed + E: The cluster could not be read from disk. There should be another + message describing more details. + +DMSDOS: check_free_sectors: wrong count %d corrected to %d + B: Looks like a minor bug. As the message tells, the problem is corrected. + +DMSDOS: clear_list_dev: Uhh, sb==NULL ... + B: Shouldn't happen. + +DMSDOS: cluster %d read error + E: There should be another message describing the error more exactly. + +DMSDOS: compression method not recognized. + E: Check filesystem. May also be caused by running incompatible/unsupported + or not yet fully supported compression software under dos. In detail, the + driver found a compressed cluster but didn't recognize the compression + header. This may even be suspicious to be a dmsdos bug. + +DMSDOS: could not find MDR signature or found more than one, mdrc=%d (ignored) + I: Debug message. It may indicate a problem with the CVF, but as this + signature is almost completely unimportant, you can most likely ignore + it. + +DMSDOS: could not guess compression method for CVF + I: There's no compressed file in your compressed filesystem that dmsdos + could analyse to determine the compression method. To avoid problems, + dmsdos has disabled compression when writing to this CVF. Specify a + compression method explicitely when mounting (comp=xxx option) or write + something (more than 1KB) to the compressed partition under dos. + +DMSDOS: counted free sectors=%d + I: The message tells how many free sectors have been counted in the CVF. + +DMSDOS: CVF almost full or highly fragmented at MDFAT level. + I: The driver warns that further write access might become dangerous. + + This indicates a really bad situation. The algorithm to find free space + in a CVF performs three trials to find an appropriate hole for new data. + All of them use different priorities to avoid MDFAT level fragmentation. + But this time it did not succeed until its last trial. That means + additional MDFAT level fragmentation cannot be avoided any longer and the + more write access you perform the more likely the last trial is to fail. + + Watch the free space on the compressed partition. If there seems to be + enough, try to defragment the partition as described in chapter + 'defragment procedures' in file dmsdos.doc. + + This message has KERN_EMERG priority to ensure you can see it on the + screen. + +DMSDOS: CVF end padding %d sectors. + I: Size of STACVOL is stored in its header. If real file is + longer than this information, user is informed about that + unused sectors. It can happens when you copy stacvol + to other host drive. + +DMSDOS CVF-FAT extension version %d.%d.%d + I: Tells version number and compile-time options when loading dmsdos. + The term 'read-only' means that you are using a read-only dmsdos + version (write access has been turned off during dmsdos configuration). + It does *not* tell that the filesystem is mounted read-only or + read-write. + +DMSDOS: CVF full. +DMSDOS: CVF full (cannot even allocate fragmented space) +DMSDOS: CVF full or too fragmented at MDFAT level. + O/E: The message tells it all. The last write access may have failed and + data may have been lost. As you cannot tell how well the data will + compress it is dangerous to fill a CVF up to the last byte. There might + even be some errors in the filesystem now (but that's very unlikely). + + These messages have KERN_EMERG priority. They indicate that something + dangerous has happened. The next time you boot dos you'd better check + the filesystem. And, of course, free up some space. + +DMSDOS: CVF has bitfat mismatches, ignored. +DMSDOS: CVF has bitfat mismatches, setting to read-only. +DMSDOS: CVF has serious errors or compatibility problems, setting to read-only. + I: The driver has detected an error in the filesystem and has either + ignored the error or set the filesystem to read-only mode. There should + be another message describing the error. In rare cases, the error may + be caused by compatibility problems. + +DMSDOS: CVF has FAT32 signature, not mounted. Please report this. + O: Compressed FAT32 is unsupported. I've never seen it, but I may start + to write support for it if someone reports that a strange M$ product + really creates compressed FAT32 filesystems. I don't believe so. + +DMSDOS: CVF is in doublespace format (version 1). +DMSDOS: CVF is in drivespace 3 format. +DMSDOS: CVF is in drivespace format (version 2). +DMSDOS: CVF is in stacker 3 format. +DMSDOS: CVF is in stacker 4 format. + I: The driver reports what it has detected. + +DMSDOS: CVF is in unknown (new?) format, please report. + I: The driver didn't recognize the CVF format after verifying that it is + indeed a CVF. Watch for further messages. You'd better not write to + the partition :) + +DMSDOS: CVF is shorter about %d sectors. Mounting READ ONLY. + E: Error in stacker filesystem. Check under dos. + Real length of STACVOL is smaller, than that stored in stacvol + header. Stacvol is probably corrupted. + +DMSDOS: CVF on device 0x%x unmounted. + I: The message tells it all. + +DMSDOS: CVF set to read-only. + I: The message tells it all. + +DMSDOS: daemon is lying about its pid + O/B: Either someone is trying to compromise system security by misusing + dmsdos ioctls (without success) or the external daemon has a bug. + +DMSDOS: daemon_write_cluster: ch==NULL + B: Shouldn't happen. + +DMSDOS: Data are not SQ compressed + E: Error in drivespace 3 filesystem (invalid data?), check under Win95. + +DMSDOS: dataend corrected due to MDR signature old=%d new=%d + I: Due to a problem with the loopback block device - it only gives size + information in units of 1K blocks and not in 512 byte sectors - dmsdos + cannot determine the exact end of the CVF. So, it tries to guess where + the real end of the compressed partition is. This guess is verified by + searching the MDR signature, which is typically located in the last + sector of the CVF. If the driver has guessed the wrong end, this is + automatically corrected and this message is logged. In theory, maximum + error can be 1 sector. + +DMSDOS: dbl_bitfat_value: version not found?? cannot happen +DMSDOS: dbl_mdfat_value: unknown version?? This is a bug. + B: Shouldn't happen. + +DMSDOS: dbl_replace_existing_cluster: checking old fraglist: first fragment wrong in cluster %d + E: Filesystem error in drivespace 3 volume. Check under Win95. + +DMSDOS: dbl_replace_existing_cluster: This is a bug - reboot and check filesystem + B: Uhh. Cut'n'paste your logfile and mail it to me. This is a serious bug. + +DMSDOS: dblspace_fat_access: READ-ONLY filesystem + O: When the filesystem suddenly is set read-only (e.g. because of a + problem) some kernel functions seem not to obey the RO flag. In this + case, the dmsdos driver itself rejects the command in order to + avoid trashing the filesystem. There's notably a problem with the + FAT driver, which seems to poke around in the FAT randomly after having + just set the filesystem to read-only mode by a filesystem panic (argh). + The reason is unknown (well, it doesn't seem to be really in the FAT + driver). To be safe, dmsdos just doesn't allow something to change the + FAT in that case. + +DMSDOS: dblspace_mark_buffer_dirty: READ-ONLY filesystem + O: When the filesystem suddenly is set read-only (e.g. because of a + problem) some kernel functions seem not to obey the RO flag. In this + case, the dmsdos driver itself rejects the command in order to + avoid trashing the filesystem. + +DMSDOS: decompression of cluster %d in CVF failed. + E/B: There's an error in the compressed data (there should be another + message describing what exactly failed). The next time you boot dos you + should check it (including the surface test since only this test finds + compression errors). If it is okay under dos/win95 but still fails + under dmsdos, *please* send a bug report (best with raw cluster data + extracted by using duitl). + +DMSDOS: dec_sq: submethod not tested - raw read + ?: Maybe a drivespace 3 problem. Check under Win95. + +DMSDOS: delete_cache_cluster: get_ch returned NULL + ?: Shouldn't happen. This message means that the cluster couldn't be + locked during deletion. So it's deleted without locking. So what. + +DMSDOS: Deleted clusters found, removing... + I: Cleaning up stacker filesystem. + +DMSDOS: dfat cache: + I: Debug message. Ignore. + +DMSDOS: dirty cluster %d on dev 0x%x removed, data are lost + O/E: The driver is giving up that cluster. Writing has failed more than + 5 times. YOU HAVE BEEN WARNED 5 TIMES! In order to prevent a system + lockup caused by endless loops the driver has thrown the data away. + THE DATA ARE DEFINITIVELY LOST NOW. Very likely the filesystem is + damaged now and a part of a directory is also lost. + The driver sets the filesystem to read-only mode (printing it in the + next message). This is a really bad situation. You'd better boot Dos + and have the CVF maintainance software repair what can be repared. + If the CVF repair tool permits it, save all lost clusters - they are + likely to contain some of the lost files. + +DMSDOS: Dirty virtual sector cannot be written - FILESYSTEM DAMAGE POSSIBLE! Trying to delay write. + O: THE FILESYSTEM IS FULL! It's even so full that some clusters cannot + be written back to the disk (they're kept in memory instead). Free up + some space IMMEDIATELY! Otherwise your filesystem will be destroyed + on unmount. After some warnings (the number depends on your cache size) + the driver will give up and stop the system with a panic. + +DMSDOS: dos max_cluster=%d too large, cutting to %d. + E?:Seems to be a filesystem error. Verify that dmsdos recognized the FAT + bit size correctly. If it hasn't, this is a bug. + +DMSDOS: error in DS-0-x compressed data. + E: There's an error in the filesystem. The next time you boot dos you + should check it (including the surface test). + +DMSDOS: error in JM-0-x compressed data. + E: There's an error in the filesystem. The next time you boot dos you + should check it (including the surface test). + +DMSDOS: Error while reading an mmap file %d <> %d + E: There should be another message describing the error more exactly. + +DMSDOS: evaluate_option: loglevel set to 0x%lx. +DMSDOS: evaluate_option: speedup set to 0x%lx. + I: The messages tell it all. + +DMSDOS: exit_daemon: counter<0 ??? + B: Shouldn't happen. + +DMSDOS: FAT bit size not recognized, guessed %d bit + I: The driver did not find a valid FAT size entry in the CVF super block + and the emulated boot block, so it guessed the FAT size according to the + CVF size. This may indicate a minor filesystem error or incompatibility. + + Warning: There's a small chance to guess the wrong FAT bit size, so be + careful. + + Note: This message may appear on all stacker filesystems since they + don't seem to have a FATxx signature in the bootblock. + +DMSDOS: FAT crosslink or loop in CVF detected (cluster %d), giving up. + E: There's a FAT level error in your compressed filesystem. This is + a serious error you should immediately repair by running dos scandisk. + + Note: Verify with dutil that dmsdos has recognized the right FAT + bit size. If it hasn't, this is a bug. + +DMSDOS: FAT size does not match cluster count. Mounting READ ONLY. + E: Error in stacker filesystem. + +DMSDOS: file_mmap_nopage: no memory! + O: Shouldn't happen. + +DMSDOS: file_read: inode = NULL, rejected. +DMSDOS: file_read: mode = %07o, rejected. + B: Maybe a bug in the FAT driver or the rest of the kernel. + +DMSDOS: file_readx: FAT mismatches file size for ino=%ld + E/B: FAT error or bug. + +DMSDOS: filesystem on dev 0x%x probably damaged, set to READ-ONLY mode + E: The message tells it all. You'd better boot Dos and have your CVF + maintainance software repair the CVF. + +DMSDOS: file_write: READ-ONLY filesystem + O: When the filesystem suddenly is set read-only (e.g. because of a + problem) some kernel functions seem not to obey the RO flag. In this + case, the dmsdos driver itself rejects the command in order to + avoid trashing the filesystem. + +DMSDOS: find_free_bitfat: free sectors=%d, cannot believe this. Counting... + B: Something is wrong with the free sector count. The driver tries to + correct this by counting the free sectors again. This is a minor bug. + +DMSDOS: find_free_bitfat returned sector %d size %d but they are not all free! + B: Cannot happen. (Uh, oh... happened in dmsdos <= 0.6.3. Don't use them.) + +DMSDOS: fraglist!=mde cluster %d sector %d!=%ld or count %d!=%d + E: Error in drivespace 3 filesystem. Check under Win95. + +DMSDOS: free_ccache_dev: oh oh, freeing busy cluster... + B: Bug in cluster cache code. Expect a crash or hang soon. Please report. + +DMSDOS: free_cluster_sectors: error in fragmentation list in cluster %d +DMSDOS: free_cluster_sectors: first fragment wrong in cluster %d +DMSDOS: free_cluster_sectors: fragmentation list unreadable in cluster %d + E: Error in drivespace 3 filesystem. Check under Win95. + +DMSDOS: free sectors=%d + I: The number of free sectors is displayed. + +DMSDOS: free sector finding statistics: + I: Debug message, ignore. + +DMSDOS: get_ch: actual looks modified ARGHHH, retrying +DMSDOS: get_ch: free looks modified ARGHHH, retrying + I: Debug message. Ignore. (This messages may be important to verify that + new SMP safe cluster locking works correctly.) + +DMSDOS: get_ch: free->c_count!=0 + B: Shouldn't happen. Indicates a bug (probably leak) in cluster caching + code. + +DMSDOS: get_ch: max_retries reached, breaking loop. This may be a bug. + B?: Shouldn't happen. Indicates that the driver broke an endless loop + to prevent a complete system hang. If you haven't run horribly out + of memory and you aren't loading the system to death this is an + extremely unlikely situation. Very suspicious to be a bug. + +DMSDOS: get_ch: oldest->c_count!=0 + B: Shouldn't happen. Indicates a bug (probably leak) in cluster caching + code. + +DMSDOS: get_cluster failed (FAT problem ?) + O/E: The FAT driver returned an error. Maybe a problem with the FAT. + +DMSDOS: giving up after %d errors. There may be more errors. + I/E: The filesystem check tries to continue after the first error was + detected, but it definitely stops after 20 errors in order to prevent + your syslog from becoming larger than the errorneous filesystem..... + +DMSDOS: illegal dfat access (cluster=%d max_cluster2=%d) + ?: Bug or filesystem error. + +DMSDOS: illegal fragcount in cluster %d + E: Error in drivespace 3 filesystem. Check it under Win95. + +DMSDOS: illegal mdfat access (cluster=%d max_cluster2=%d) + ?: Bug or filesystem error. + +DMSDOS: illegal virtual sector %d, can't map to real sector + B: Problem with virtual sector mapping. + +DMSDOS: Inconsistent first data sector number. Mounting READ ONLY. +DMSDOS: Inconsistent sector length + E: Error in stacker filesystem. + +DMSDOS: init_daemon: daemon already present + I: Debug message, ignore. + +DMSDOS: Interesting MDFAT non-lin subalocation (cluster %d) + I: Will be disabled, because it is normal for stacker 4. + +DMSDOS: ioctl: D_READ: no memory! + O: Lack of system memory. Can be ignored. (*)[see below] + +DMSDOS: ioctl: D_READ: read_cluster failed! + E: Seems to be an error in your filesystem (the dmsdos daemon tried to + read some data but read access failed). There should be at least one + preceeding message that describes the error more exactly. + +DMSDOS: ioctl: D_WRITE: no memory! + O: Lack of system memory. Can be ignored. (*)[see below] + +DMSDOS: ioctl: loglevel set to 0x%lx. + I: The message tells it all. + +DMSDOS: ioctl: read_cluster: no memory! + O: The driver could not read a cluster because of lack of memory. Don't run + so much applications at the same time, increase your swap space or add + memory to your machine. (Shouldn't happen.) (*)[see below] + +DMSDOS: ioctl: speedup set to 0x%lx. + I: The message tells it all. + +DMSDOS: killing internal daemon... + I: Debug message. Ignore. + +DMSDOS: list statistics: + I: Debug message. Ignore. + +DMSDOS: lock_ch: count=0! This is a bug. + P: Serious bug in cluster caching code. Please report. The system might + even hang completely now. + +DMSDOS: log_ccache_statistics: cannot happen. +DMSDOS: log_list_statistics: cannot happen. + B: Bug, please report. + +DMSDOS: MDFAT bad allocation (cluster %d) + E: Error in stacker filesystem. + +DMSDOS: mdfat cache: + I: Debug message, ignore. + +DMSDOS: MDFAT crosslink in CVF detected (cluster %d) +DMSDOS: MDFAT crosslink detected (cluster %d) + E: There's a MDFAT level crosslink in your compressed partition. This is a + serious error you should immediately repair by running dos scandisk. + +DMSDOS: MDFAT entry invalid (cluster %d, sect %d) +DMSDOS: MDFAT entry invalid in CVF (cluster %d) +DMSDOS: MDFAT entry invalid in CVF (fragmented cluster %d fragpnt %d) + E: There's a problem with your compressed partition. Dmsdos ignores + invalid entries if they are assigned to a file (the file will be + unreadable). + +DMSDOS: MDFAT-level dead sectors found in CVF (cluster %d) +DMSDOS: MDFAT-level dead sectors found, removing... + E/I: The driver has found sectors in the filesystem that belong to a + non-existing cluster. Depending on the repair flag, it tries to + repair the problem. + +DMSDOS: MDR signature found at sector %d + I: Debug message. Usually shows the number of the last sector in the CVF. + +DMSDOS: MDR test breaks at i=%d + I: Debug message that is produced during CVF size test. There's very + likely also a message 'access beyond end of device' logged just before + this message. It's part of the MDR test to provoke an access beyond + the device end. You can safely ignore both messages. + +DMSDOS: mount_dblspace: out of memory + O: Shouldn't happen. + +DMSDOS: mounting CVF on device 0x%x %s... + I: The message tells it all. + +DMSDOS: mount_stacker: out of memory + O: Shouldn't happen. + +DMSDOS: MOVEBACK ioctl has gone + O: Then don't use it. + +DMSDOS: MSDBL/MSDSP signature not found, CVF skipped + E: This file doesn't seem to be really a CVF. The driver refuses, for + example, to mount your phone directory that has been renamed to + something like dblspace.001. Check the compressed filesystem under + dos (if dos refuses to boot because it runs into a similar problem, + use a dos boot disk and hold down CTRL and F8 during boot process). + +DMSDOS: no memory for decompression! + O: The driver could not decompress a cluster because of lack of memory. + Don't run so much applications at the same time, increase your swap space + or add memory to your machine. (Shouldn't happen.) (*)[see below] + +DMSDOS: page-aligned memory returned by kmalloc - please disable XMALLOC + B: The xmalloc allocation mechanism is broken. This may be caused by + using a very new kernel. You can still use dmsdos, but you are strongly + encouraged to disable xmalloc: rerun the dmsdos configuration, disable + advanced memory management and recompile. Please send also a problem + report so I can start looking for what broke xmalloc in your kernel. + +DMSDOS: part 1 of filesystem check failed, aborting. +DMSDOS: part 2 of filesystem check failed, aborting. + I: The message tells it all. + +DMSDOS: read BITFAT state error + E: Problem with stacker filesystem. Probably low level host drive IO + error. + +DMSDOS: read_cluster: illegal cvf version flag! + B: Bug, please report. + +DMSDOS: read_cluster: mdfat sectors > sectperclust, cutting + E: There's an error in the filesystem. The next time you boot dos you + should check it (including the surface test). (This indicates a + corrupted MDFAT, may be suspicious to be a dmsdos write access bug.) + +DMSDOS: read_file bug: f_pos not cluster-aligned + B: ??? + +DMSDOS: read_fragments: cluster does not look fragmented! +DMSDOS: read_fragments failed! +DMSDOS: read_fragments: safety_counter exceeds membytes! + E: Error in drivespace 3 filesystem. Check under Win95. + +DMSDOS: read_fragments: size limit reached. + I: Debug message. Ignore. + +DMSDOS: read_the_page: no memory! + O: Shouldn't happen. + +DMSDOS: read_virtual_sector: no memory! +DMSDOS: read_virtual_sector: read_cluster failed! + E/I: Problem with virtual sector handling. + +DMSDOS: READ/WRITE DIRENTRY ioctl has gone +DMSDOS: RECOMPRESS ioctl has gone + O: So don't use them. + +DMSDOS: sd4_comp: Compression ends with mismash +DMSDOS: sd4_comp: Hufman code leakage in table 1 +DMSDOS: sd4_comp: Hufman code leakage in table 2 +DMSDOS: sd4_huffman: Problems with number of bits + I: Huffman code length of some character is longer + than 15 bits in STAC4 compression. Code will be recomputed + with little worse compression ratio. + Message may be disabled in future. + +DMSDOS: sd4_decomp: no memory! + O: Data cannot be decompressed, memory is low. (See below for 'no memory' + messages.) + +DMSDOS: sd4_decomp: Magic = %X => error! +DMSDOS: sd4_decomp: Table 2 consistency check !!!! +DMSDOS: sd4_decomp: Table 1 consistency check !!!! +DMSDOS: sd4_decomp: Table 1 error +DMSDOS: sd4_decomp: Under !!! +DMSDOS: sd4_decomp: Over !!!! +DMSDOS: sd4_decomp: End read %X and should be %X +DMSDOS: sd4_decomp: Error end token %X + E: Data cannot be decompressed. + Filesystem probably damaged. Run DOS stac/check. + +DMSDOS: SETMAXCLUSTER ioctl has gone. + O: This ioctl command has turned out to be too dangerous. Use the native + CVF maintainance tools that came with your CVF package under Dos/Win95. + +DMSDOS: set_maxcluster %d refused: cluster %d in use + O: You tried to specify a cluster limit that is too low via the + 'setmaxcluster' command of dutil. This message also displays the + currently lowest possible value ('cluster %d in use'). Your command has + been skipped. + +DMSDOS: set_maxcluster refused - CVF is not in doublespace or drivespace<=2 format. + O: The setmaxcluster command is intended as workaround for a bug in + dos scandisk. You can only use it on CVFs that have been created by + a doublespace version that has this bug. + +DMSDOS: strange version flag %d, assuming 0. + I/E: The version flag in the CVF header contains garbage. Either it's + destroyed or you have found a new doublespace/drivespace version. In + the latter case please let me know. + +DMSDOS: simple_check aborted (no memory) + O: Lack of kernel memory caused the dmsdos driver to abort the filesystem + check. Shouldn't happen. (*)[see below] + +DMSDOS: simple_check: BITFAT abnormal state +DMSDOS: simple_check: BITFAT mounted/dirty +DMSDOS: simple_check: BITFAT state error + IE: The BITFAT/allocation map of the stacker filesystem is marked as + 'not up-to-date'. This may have been caused by a dos crash or reset. + The filesystem can only be used read-only. You can repair it by + booting DOS (stacker will show its "updating allocation map" + message) or use dmsdos' mount option 'bitfaterrs=repair' or use dutil. + +DMSDOS: simple_check: read BITFAT sumary error + E: Error in stacker filesystem. + +DMSDOS: simple_check: MDFAT+BITFAT test skipped (no memory) + O: Lack of kernel memory caused the dmsdos driver to skip the MDFAT and + BITFAT test. Since this test may require up to 1 MB kernel memory + (depending on the partition size), this message may occur occasionally. + You needn't be concerned about this. (*)[see below] + +DMSDOS: sq_comp: ERROR: Processed only %d bytes !!!!!! +DMSDOS: sq_comp: Huffman code leakage in table 1 +DMSDOS: sq_comp: Huffman code leakage in table 2 +DMSDOS: sq_comp: Huffman code leakage in table 3 +DMSDOS: sq_huffman: Problems with number of bits + I: SQ compression problem. Data will be compressed again with lower + compression ratio or written without compression. + +DMSDOS: sq_comp: Not enough memory + O: The data written uncompressed due to lack of memory for compression. + +DMSDOS: SQ-0-0 decompression failed. +DMSDOS: sq_dec: huff BAD last token %x +DMSDOS: sq_dec: huff count_1 too big +DMSDOS: sq_dec: huff count_2 too big +DMSDOS: sq_dec: huff error in char and len table +DMSDOS: sq_dec: huff error in helper table +DMSDOS: sq_dec: huff error in offset table +DMSDOS: sq_dec: huff offset OVER +DMSDOS: sq_dec: huff offset UNDER +DMSDOS: sq_dec: out of memory! +DMSDOS: sq_dec: submethod not tested - fixed huffman +DMSDOS: sq_dec: unknown submethod - 3 + E/B: Error in SQ compressed data (drivespace 3). Check under Win95. + I've been reported about SQ decompression problems when the files + even decompress well under Win95. There seems to be a problem + somewhere in the SQ decompression code, but I'm unable to fix it. + +DMSDOS: stac3_decomp: char repeat overrun! +DMSDOS: stac3_decomp: end token 0x%02X +DMSDOS: stac3_decomp: Illegal back pointer length 0x%x at pos 0x%x->0x%x +DMSDOS: stac3_decomp: Multi rep overrun 0x%x at pos 0x%x->0x%x +DMSDOS: stac3_decomp: ReadNC error! +DMSDOS: stac3_decomp: Unknown token %d on pos 0x%X->0x%X +DMSDOS: stac3_decomp: xor sum error! + E: Looks like your stacker filesystem is corrupt. The data of a file + couldn't be decompressed. + +DMSDOS: stac3_decomp: Multi rep:(%dx %d) +DMSDOS: stac3_decomp: Rep:(%dx) + I: Debug messages. Ignore. + +DMSDOS: stac_cwalk_init: bad bytes_in_cluster %d +DMSDOS: stac_cwalk_init: count = %d < 0 in long subalocated +DMSDOS: stac_cwalk_init: count = %d < 0 in short subalocated +DMSDOS: stac_cwalk_init: fragment signature not found cluster=%d +DMSDOS: stac_cwalk_init: sector count mismash fragmented cluster=%d! +DMSDOS: stac_cwalk_init: suballocation signature not found cluster=%d +DMSDOS: stac_cwalk_init: suballocation error 1, cluster %d +DMSDOS: stac_cwalk_init: suballocation error 2, cluster %d +DMSDOS: stac_cwalk_init: suballocation error 3, cluster %d, zerro offset 0x%X 0x%X +DMSDOS: stac_cwalk_init: suballocation error 4, cluster %d +DMSDOS: stac_cwalk_init: suballocation not present, cluster %d +DMSDOS: stac_cwalk_init: too much fragmented cluster=%d! +DMSDOS: stac_cwalk_init: unknown flags 0x%2x cluster %d +DMSDOS: stac_cwalk_init: wrong cluster types for subalocation, cluster %d + E?: Looks like your stacker filesystem is corrupt. + +DMSDOS: stac_cwalk_sector: finfo==NULL, cluster %d + B: Internal bug in fragmented clusters access. + +DMSDOS: Stacker 0x1A0A signature not found + E/O: This CVF doesn't seem to be really a CVF. + +DMSDOS: Stacker sector size not 512 bytes, hmm... + E: Strange error in stacker filesystem or incompatible version (?) + +DMSDOS: STACKER signature not found + E/O: This CVF doesn't seem to be really a CVF. + +DMSDOS: stac_read_cluster: alloc error in cluster %d +DMSDOS: stac_read_cluster: decompression error cluster=%d +DMSDOS: stac_read_cluster: internal cw error 1 cluster=%d +DMSDOS: stac_read_cluster: internal cw error 2 cluster=%d + E: Error in stacker filesystem. + +DMSDOS: stac_read_cluster: no memory! + O: The driver could not read a cluster because of lack of memory. Don't run + so much applications at the same time, increase your swap space or add + memory to your machine. (Shouldn't happen.) (*)[see below] + +DMSDOS: stac_replace_existing_cluster: This is a bug - reboot and check filesystem + B: Safety check in allocation routine failed. Please send a bug report. + +DMSDOS: stac_special_free: alloc error in cluster %d + E: Problems with deleting of cluster may be caused by + - damaged filesystem + should be reported when mounted as bitfat mismashes + can be repaired by mount option bitfaterrs=repair + - dmsdos internal bug + +DMSDOS: stac_write_cluster: alloc error in cluster %d +DMSDOS: stac_write_cluster: internal cw error 1 cluster=%d +DMSDOS: stac_write_cluster: internal cw error 2 cluster=%d + E: Error in stacker filesystem. + +DMSDOS: stac_write_cluster: no memory for compression, writing uncompressed! + O: Shouldn't happen.(*)[see below] + +DMSDOS: starting internal daemon... + I: The message tells it all. + +DMSDOS: Sumary: Deleted clusters = %d +DMSDOS: Sumary: Free sectors = %d +DMSDOS: Sumary: info1 = %d +DMSDOS: Sumary: info2 = %d + I: Internal stacker messages. Information from checking of CVF. + First two are counted and next two are read from CVF. + +DMSDOS: support for doublespace/drivespace(<3) not compiled in. +DMSDOS: support for drivespace 3 not compiled in. +DMSDOS: support for stacker 3 not compiled in. +DMSDOS: support for stacker 4 not compiled in. + O: The message tells it all. Rerun the dmsdos configuration and recompile + the source if you do need support for this type of CVF. + +DMSDOS: Too many BITFAT mismatches, check aborted. +DMSDOS: Too many BITFAT mismatches in CVF, check aborted. + E: The message tells it all. + +DMSDOS: try_daemon: kill_proc daemon_pid=%d failed with error code %d, assuming daemon has died + O: The driver tried to wake up the external daemon, but it seems to be no + longer running. + +DMSDOS: try_daemon: no empty slot found, listcount corrected. + B: Shouldn't happen. + +DMSDOS: try_fragmented: cnt<0 ? This is a bug. +DMSDOS: try_fragmented: frags=%d ? Cannot happen. +DMSDOS: try_fragmented returned non-free sectors! + B: Shouldn't happen (bugs in code for writing fragmented clusters for + drivespace 3 volumes). + +DMSDOS: trying to allocate fragmented space... + I: The driver tries to create a fragmented cluster since disk space is + low or highly fragmented. + +DMSDOS: unable to read acache area=%d +DMSDOS: unable to read bitfat area %d for sector %d +DMSDOS: unable to read dfat area %d for cluster %d +DMSDOS: unable to read emulated boot block of CVF +DMSDOS: unable to read mdfat area %d for cluster %d +DMSDOS: unable to read second dfat +DMSDOS: unable to read second mdfat +DMSDOS: unable to read super block +DMSDOS: unable to read super block of CVF + E: Bad block or serious error in the underlying msdos filesystem. The + CVF or the underlying filesystem is most likely destroyed. + +DMSDOS: unable to read fragmentation list of cluster %d. + E: Error in drivespace 3 filesystem or hardware error. + +DMSDOS: unknown option %s, rejected + O: Syntax error in mount option string. + +DMSDOS: Updating BITFAT +DMSDOS: Updating BITFAT. +DMSDOS: Updating BITFAT finished +DMSDOS: version_flag=%d sectperclust=%d + I: Debug info. + +DMSDOS: vmalloc returned unaligned memory - please disable XMALLOC + P/B: The xmalloc allocation mechanism is broken. This may be caused by + using a very new kernel. The driver detected this problem and stopped + immediately to prevent memory corruption. You may have to press the + reset button after this message. You can still use dmsdos, but you must + disable xmalloc before: rerun dmsdos configuration and disable advanced + memory management. Please send also a problem report so I can start + looking for what broke xmalloc in your kernel. + +DMSDOS: write access not compiled in, ignored + I: Someone or something tried to write to a compressed partition but write + access support is not compiled in. This message always appeares + *instead* of a real write access. + + If you *want* to write to a compressed partition, you must compile + in write access support before. Rerun the dmsdos configuration + ('make config' in the src directory), recompile and reinstall dmsdos. + +DMSDOS: write_cluster: guessed 0x%08x. + I: The driver succeeded to guess the compression method and tells what + it has guessed. + +DMSDOS: write_cluster: guessing compression method... + I: The driver is analysing some clusters to determine the compression + method automatically. This kind of 'guessing' has proven to be very + reliable, so it may be a good idea to let the driver guess the + compression method instead of specifying it explicitely in a mount + option. + +DMSDOS: write_cluster: illegal cvf_version flag! + B: Looks like a bug in the dmsdos configuration (some missing or wrong + ifdefs somewhere). Let me know, please. Meanwhile, configure dmsdos + to support everything, recompile, and try again. + +DMSDOS: write_cluster: no memory for compression, writing uncompressed! + I: The driver could not compress a cluster because of lack of memory, so + it skipped compression. Don't run so much applications at the same time, + increase your swap space or add memory to your machine. (Shouldn't + happen.) (*)[see below] + +DMSDOS: write_file: ch_dirty failed! +DMSDOS: write_file: ch_noread failed +DMSDOS: write_file: ch_noread failed! + E: There should be another message describing the error. + +DMSDOS: write_file: CVF full (free sector count too low) +DMSDOS: write_file: CVF full (full flag set) + O: Guess what. To prevent filesystem corruption, the driver has stopped + write access at a cluster boundary and returned the 'no space left on + device' error code in order to give an application the chance to exit + cleanly. + + WARNING: As you cannot exactly say how well the data compress it is + dangerous to fill a CVF up to the last byte. The driver tries to stop + applications quite a time before it becomes dangerous. This message + indicates that up to now nothing dangerous has happened. + +DMSDOS: write_file: length>clustersize ??? bug !!! + B: Cannot happen. + +DMSDOS: write_file: fat_add_cluster failed + O: Filesystem is full or has run out of clusters. Run dutil to distinguish + between these two situations. In the latter case you need to boot dos + and increase the estimated compression ratio. + +DMSDOS: write_file: read_cluster failed! + E: There should be another message describing the error. + +DMSDOS: write_file: something wrong, cannot happen +DMSDOS: write_file: something's wrong, cannot happen + B: Problem with writing a file. + +DMSDOS: write_fragmented: raw_getblk sector %d failed + O/B: Low-level disk i/o problem or serious bug (may even be a bug in + the rest of the kernel). A fragmented cluster couldn't be written. + This leaves behind a damaged filesystem. You must repair it under + Win95 (if it is a real hardware error, e.g. dying disk, you might + be lost hopelessly without a backup). + +DMSDOS: unable to read emulated boot block + E: Error in filesystem, check it under dos. + +DMSDOS: zero_new_cluster: ch_noread failed??? + O/B: Don't know. Really. This shouldn't happen. + +VFS: LRU list corrupted +VFS: free list corrupted +(or similar) + P/B/O: You are very suffering from the vfat brelse bug (this is a serious + bug in some 2.2.x and maybe late 2.1.x kernels). This bug is triggered + by dmsdos. Ensure you have applied the vfat-brelse-bugfix.diff or you + are using a kernel where the bug is fixed (in 2.2.2 it is not yet fixed). + See file patches/DIFFS.TXT or INSTALL.TXT for details. + + If not, this indicates probably a leak in dmsdos virtual sector code. + Please send a bug report to the current dmsdos maintainer and be prepared + to be asked for some tests in order to track down the bug. + +(*) ['no memory' messages:] + There was a problem in previous dmsdos versions that caused these + messages rarely though there was enough memory free. I've tried to fix + it by calling another memory allocation routine in newer dmsdos + releases. It's currently unknown if the new method is better. So if + you think you receive 'no memory' messages too often, you can try to + switch the allocation mode. It's setup during dmsdos configuration + ('make config'). If you change it, please let me know your results. diff --git a/doc/patent.doc b/doc/patent.doc new file mode 100644 index 0000000..bce9427 --- /dev/null +++ b/doc/patent.doc @@ -0,0 +1,101 @@ + +Concerning the relationship between dmsdos and patents I keep receiving mail +like this one... + +> In article <[email protected]> you wrote: +> : dmsdos 0.9.0 has just been released. It supports read and write access to +> : compressed dos filesystems [...] +> +> I think you should make sure no patents are violated. I am afraid that you +> MUST get permission to distribute your code. I wish it was impossible +> to get a software patent, but unfortunately companies do receive these +> patents. There are many patents in encryption and compression technology! +> [...] +> Again, I wish it was possible/legal to write a free variant for every +> existing commercial variant. Unfortunately it is not always possible. + +Let me say some gereral things about this. The patent problem is indeed as +old as the first dmsdos version, which was released approximately three +years ago. + +First, please allow me to correct some statements that are not litterally +written above, but that sound like a misunderstanding. Dmsdos is not a +"free variant" of an "existing commercial variant". Even both "free" and +"variant" are wrong. Dmsdos is not free, but copyrighted and distributed +under the GPL. + +Dmsdos also is not a "variant" of some commercially available software +package. It is just a tool that makes it possible to read from and write +to some variants of compressed dos filesystems. Dmsdos is surely not a +filesystem compression software package like stacker and doublespace, as +such a package needs a lot of more software e.g. all kinds of creation and +maintainance tools. I know it would be nice to have them under Linux, but +I won't write them for well known reasons. + +If you follow the dmsdos history, i.e. the dmsdos documentation of older +releases, there have always been some comments on the possibility of patent +problems mentioned in the documentation, and thus some features users liked +to have in dmsdos were not implemented. Not because I knew they are +forbidden by patent but because I thought they *might* be covered by a +patent. + +Yes, I did contact the respective companies (no need to say which ones :) ) +for legal issues and, of course, I asked whether they were willing to help +developing the code by providing documentation about their filesystem. +So what? When I really had luck and got an answer, it was of no value. A +stripped down version of one answer was published once in the dmsdos +documentation. I removed the name of the company and the name of the person +since I didn't want to blame one person. But I wanted to show the level +of interest of these companies - on the one hand in support and cooperation, +and on the other hand in a Linux version of their code. It was absolutely +zero. + +So I did my own research. It was surely not exhaustive. Patents are things +for lawyers, but I'm not a lawyer myself and I don't have the money to +get a bunch of lawyers study all the software patents concerning data +compression. Furthermore, I don't earn any penny with dmsdos. + +It lead to the result that there are a lot of patented compression +algorithms. Also the compression algorithms that the original dos software +uses for filesystem compression are covered by patents. So what. Dmsdos +doesn't use them. Dmsdos was developed without official documentation, and +it turned out that its compression algorithm even reached a higher +compression ratio (but was much slower). + +I still tried to contact the companies, rarely, but it became more and more +boring. I must admit that I gave it up some time ago. I also must admit +that I didn't add all the patent problem related stuff to the dmsdos +documentation when I rewrote it some day. I considered it simply dead. +This implies that some features are still missing in dmsdos and will +probably never be added because I don't just want to be exposed to the +risk of violating a software patent and provoking a company owning it. +The community of dmsdos users on the net seem to have accepted this. + +I also must say that dmsdos is not at all fully my own work. I just happen +to maintain the code currently, and I'm not doing this on my own. The +documentation used to implement dmsdos came from a lot of people on the +net and even from a previous sample implementation(*) that was released under +the GPL. If you know the GPL you also know that it has a very restrictive +patent section, so I considered this quite safe. + +In fact, the compression and decompression routines in dmsdos are something +like a collection of parts of free or GPL'd software. Most of them have +meanwhile been rewritten from scratch for better performance. They use very +common compression techniques. And you don't receive a patent for something +that is well known. You can receive a patent, e.g. for a special, highly +optimized algorithm, but, let me repeat this, dmsdos does not use any +patented compression algorithms. + +So what can I do? Just throw away the dmsdos code and remove it from +the servers? This is like a snail going back into its house and staying +there though nothing is happening outside. Just continue trying to contact +the companies? I'm bored by their answers if I happen to get one. Sorry. +Just giving dmsdos maintainance into the hand of someone else? Heh. That +would solve the problem probably for me, but not for others. + +If you just happen to know more about the patent situation than me, please +let me know. + + +(*) ftp://sunsite.unc.edu/pub/Linux/system/Filesystems/dosfs/thsfs.tgz + written 1994 by Thomas Scheuermann (current email address unknown) diff --git a/doc/testing.doc b/doc/testing.doc new file mode 100644 index 0000000..e8b342b --- /dev/null +++ b/doc/testing.doc @@ -0,0 +1,68 @@ + +Testing of read access with Dosemu +================================== + + +I describe the method I use for my drives, you may need to change some names. +I have a DOS filesystem on /dev/hda1, I usually mount it on /DOS. +This drive is seen as c: from DOS when starting. +Stacvol file is c:stacvol.dsk. When config.sys is executed +stacvol.dsk is swapped and becomes c: and old c: is d:. + +Prepare dmsdos package in /usr/local/src/dmsdos-x.y.z/src + rmmod dmsdos # for changes from next compilation + make clean + make + insmod dmsdos + cd /DOS + mount -r -t msdos -o loop stacvol.dsk /mnt # important: mount READ-ONLY!!! + +Now you should see the root of your DOS drive in /DOS directory and +under /mnt the root of your stacker drive. +In the syslog you can see the parameters of all mounted stacker drives. +If you have dutil compiled, you can proceed the next test + + dutil /mnt checkfs + +If there are errors you can see them in the syslog as DMSDOS: ... +Now you can make the read test. +Configure Dosemu for readonly access to the full hard drive or partition. + + disk { partition "/dev/hda1" readonly } # must be first to boot from it + disk { image "/var/lib/dosemu/hdimage" } # or something similar + # you need this to have access to dosemu utilities + +You must include somewhere in config.sys line + + device=<hdimage_drive>:\emufs.sys / + +Now run Dosemu by command + + dos -C + +If it does not work try to find out if stacker can see its stackvols. +There may be differences if you have some other organization of the drives +under DOS. +Now you can see or change redirections to the Linux filesystem + + <hdimage_drive>:lredir + +For comparing you can use the dos utility cmpdisk.exe, which tries read all +files from two drives. One should be the drive managed by regular +dos stacker and the second by the Linux dmsdos driver. + + cd <stac_drive>:\ + cd <redirected_linux_drive>:\mnt + cmpdisk <stac_drive>: <redirected_linux_drive>: + +If some file is read differently from one drive, cmpdisk waits +for a keystroke. Then it continues. Parameters of cmpdisk can +be in form: + + <drive_letter>:[\]<directory>\ + +for example + + E:\mnt\ + +Any differences are suspected to be dmsdos bugs! diff --git a/doc/troubleshooting.doc b/doc/troubleshooting.doc new file mode 100644 index 0000000..5d15ad4 --- /dev/null +++ b/doc/troubleshooting.doc @@ -0,0 +1,290 @@ +troubleshooting.doc + +This file contains solutions to common problems with the dmsdos driver. +It is mostly based on the feedback of dmsdos users that have sent mail to me +for some hints (thanks a lot to all of you!). + +------------------------------------------------------------------------------ + + If you have problems, you may have found a bug. But check the following + list before (it is a collection of problems I have been reported and, I + think, can be solved easily). See also file BUGS for a list of known bugs. + + *** See also your kernel log for 'DMSDOS:...' messages. Each message is + explained in file messages.doc. There you can find hints how to + solve the problems, too. + +- It does not install/compile! + + Maybe the cvf.diff did not apply correctly. Check for *.rej files in + your kernel sources (only for the 2.0.xx kernels). + + In some 2.1.xx kernels umsdos is broken. Get at least 2.1.94 if you want + to use umsdos together with dmsdos. + + Depending on the dmsdos configuration the code might fail to compile. + Critical configuration options are: + + * advanced memory management (switch it off if problems occur) + * writable mmap (switch it *on* for latest 2.1.xx kernels) + + Please keep in mind that 2.1.xx kernels are still changing quite fast. + I cannot test every configuration option against each kernel... + +- The module doesn't load. It fails with a message about undefined symbols. + + The dmsdos module requires fat filesystem support. If the fat module is + not loaded and not part of the kernel the dmsdos module refuses to load + and complains about missing symbols. Since kernel 2.0.34, fat additionally + depends on nls support. If it is the UMSDOS module that doesn't load, + see also the next problem :) + +- The UMSDOS module doesn't load though fat and msdos support is present. + It complains about missing symbols 'fat_readpage' and 'fat_dir_ioctl'. + + This is a known bug in some older cvf.diffs from alpha test versions. + I forgot to add some symbols to the export list in older cvf.diffs for + kernel 2.0.33. This bug also went in 2.1.80 when CVF-FAT was included. + In order to fix the problem, edit file linux/fs/fat/fatfs_syms.c and + append the missing symbols 'fat_readpage' and 'fat_dir_ioctl' to the + export list. Look how the other symbols are listed there and put in the + missing ones in the same way. Then recompile the fat module (or the + kernel if your fat support isn't modular). + +- It causes a kernel panic with the message "VFS: LRU list corrupted". + + You are very suffering from the vfat brelse bug (this is a serious + bug in some 2.2.x and maybe late 2.1.x kernels). This bug is triggered + by dmsdos. Ensure you have applied the vfat-brelse-bugfix.diff. + See file patches/DIFFS.TXT or INSTALL.TXT for details. + + If not, this indicates probably a leak in dmsdos virtual sector code. + Please send a bug report to the current dmsdos maintainer and be prepared + to be asked for some tests in order to track down the bug. + +- It crashes and writes a register dump to the syslog. + + If you did not load the module with the -m flag this time, you are out + of luck here. Please load the module with the -m flag (this prints a + symbol table) and redirect the output into a file. This symbol table is + required to analyse the crash. The symbol table usually differs from one + insmod to another depending on free memory pages, so you cannot just + use an older one :) Then try to reproduce the problem. + + This smells like a bug - most likely. To be sure that this is really a + dmsdos related problem please read the README file in the Linux kernel + sources (/usr/src/linux/README). There's a procedure explained how to + obtain valuable information from the register dump in the syslog, in + particular, how to find out in which function the crash occured. Please + note that the register dump is absolutely system dependent and thus must + be analysed and interpreted by *you* (it's really not difficult, but it must + be done carefully - otherwise it's meaningless). Just use the symbol table + the insmod -m command created *before* the crash. + + If it turns out to be a dmsdos function, you have discovered a serious bug + that should be fixed. Please send a bug report in that case. Please also + explain how to reproduce the problem with simple commands like 'ls', + 'cat' or 'touch', for example. + +- It hangs without any comments. + + Can you switch to another virtual console and enter commands there? What + does 'ps' tell about the hanging process? Can you kill it? If the answer + to all these is no, this may be a bug. + + Can you reproduce the problem with simple commands like 'ls' or 'cat'? + + Please note that write access to a compressed partition may be awfully + slow. This depends on the compression level and on the fragmentation of + the compressed filesystem. Such a situation might be interpreted as a hang + though it isn't really one (especially on a 386SX16 machine...). + + Also reading a large file or a huge directory might cause a system freeze + for a short time. This is especially true for the /dev directory (given + that you copied it to a compressed partition...) + +- After mounting the CVF I only see garbage when I do 'ls /_mountpoint_'. + + Are you sure you loaded the dmsdos module? The plain fat driver (without + the dmsdos module) mounts some compressed volumes without complaining, but + it really can't deal with them and logs a lot of errors when trying to + read, for example, a directory on that partition. Since the mount command + is exactly the same this might indeed happen accidentially. + + Take a look at your system log (usually in /var/log/messages). If you + don't see DMSDOS messages - unmount the filesystem, load the dmsdos module + and try again. + + If you are sure you have loaded the module (try 'lsmod') dmsdos auto + detection may have failed. Try again with the appropriate cvf_format=xxx + mount option, i.e. cvf_format=dblspace (for M$ doublespace and drivespace) + or cvf_format=stacker (for Stacker 3 and 4). + + If you do see DMSDOS messages, there are very likely much of them. If + the dmsdos driver cannot deal correctly with a filesystem it becomes + very noisy and sets the partition to read-only mode. In that case, please + let me know. You might have a new dblspace or stacker version that + should be supported too. + +- I cannot write to files, delete files, create directories ... in the + compressed subdirectories. + + Lots of possibilities here -- watch your kernel log for a DMSDOS error + message that may explain what went wrong... + * Have you disabled write access at compile time? Then you'll find a + "write access not compiled in" message. + * Did you mount read-only (option ro)? + * Is your compressed partition almost full? The driver refuses to allocate + new clusters on a partition that is almost full when it thinks further + write access may be dangerous and might cause data loss. Delete some + files and retry. + * The compressed partition may have errors. The driver sets errorneous + compressed partitions to read-only immediately after mounting them. + However, you can allow write access again by setting the comp parameter + via dutil (example: 'dutil /DOS/dblspace.001 setcomp guess'). Of course, + it is not recommended to write to errorneous partitions..... + +- When trying to write to a compressed filesystem I receive I/O errors. + + See previous problem. If you think it's another reason, run dutil to check + whether the filesystem is full or too fragmented. If the compressed + filesystem is read-only, all write access to the compressed filesystem is + refused. There is, of course, no real I/O error. The dmsdos driver returns + the 'permission denied' or 'no space left on device' error code depending + on the situation, but some applications simply claim 'I/O error' instead. + +- When trying to write to the compressed partition, I receive "no space left + on device" errors though dutil says there's *a lot* of free space (>100KB). + + That depends. 100KB is not much. If the filesystem does not allow + fragmented data dmsdos reserves the space of 10 clusters for the case of + emergency (just imagine that data that compress well can be replaced by + data that don't compress at all and need more physical space). For a CVF + with 8KB cluster size it's 80KB, for a CVF with 32KB cluster size it's + 320KB. You cannot use this space for regular data. If the filesystem + supports fragmented data (e.g. drivespace 3, stacker 4) it's enough to + reserve space for one fragment of maximum size (drivespace 3: 32.5KB, + stacker 4: fragment support for write access is not yet implemented, + so size is 10 clusters). + + There are some other possibilities. You may have run out of clusters. The + driver ensures that you don't get more clusters than dos would give to + you (otherwise dos' scandisk.exe would *destroy* the compressed partition + the next time it checks it by inventing some strange errors and trying to + correct them [arghhhh... seems to be a scandisk bug]). This can be fixed + by booting dos, running dblspace.exe or drvspace.exe, and setting the + compression ratio to the value the program suggests. (The procedure for + Stacker should be similar.) Don't be surprised if it suggests high + compression rates you aren't used to from dos - it's because dmsdos + compresses a little better than dos itself. (In fact, you needn't + believe the value dos claims - it's exaggerating sometimes....) + + The third possibility is that your compressed partitiion has become too + fragmented at internal MDFAT level. Dos 6.0-6.22 Doublespace/Drivespace + and Win95 Doublespace/Drivespace (*not* drivespace 3) and Stacker 3 are + known to suffer extremely from internal MDFAT level fragmentation. Note + that this kind of fragmentation is different from usual FAT level + fragmentation and can not be repaired by those popular FAT defragmentation + programs. Use the CVF maintainance tools that came with your CVF software + to defragment the CVF. + +- A file I want to read appears to be empty but the directory says it is not + empty. + + There was a problem decompressing the file. See your kernel messages + (/var/log/messages). The first thing to do is to boot dos and run the dos + filesystem checker on the compressed partition (do not skip the surface test + since only this test finds erros in compressed data). After that, reboot + Linux and retry. If the problem did not disappear this may be a real + dmsdos bug and should be emailed to the author. + +- I mounted a compressed floppy at /mnt as type umsdos and tried to run + umssync on the disk. It failed with an error message and now some or all + files are gone (help)! + + You cannot umssync a compressed floppy without freeing up some space for + the umsdos special file --linux-.--- (the real msdos part of a compressed + floppy is always totally full). + + But you have luck, the files are not gone at all. Unmount the floppy and + mount it as type msdos. Then remove the (empty) file --linux-.--- and the + 'readme' or 'readthis' or whatoever (it contains a message that this disk + is compressed and that you must run Micro$ doublespace to read it ... + very interesting - and not true at all: you can use Linux and dmsdos + instead :-) ). Now there should be some bytes free for umsdos. Unmount + the floppy again, mount it as type umsdos and retry. If the problem + appeared inside the compressed partition, also mount as type msdos, remove + --linux-.--- where files were gone, free up some space, mount again as type + umsdos, and retry running umssync. + + This problem might be considered a umsdos caveat. + +- The utility dutil reports a different compression ratio and different free + space than dos. + + The utility uses another method to calculate the compression ratio and the + amount of free space. Well, dos displays a dream ratio, dutil exactly + determines how much the files shrunk on compression. Dos' ratio is higher + because it compares the space the files *have* allocated with the space + the files *would have* allocated on an uncompressed partition with an 8 KB + (32KB under drivespace 3) cluster size. The slack at the end of the files + causes the difference (especially on small files), but saving wasted space + is not the same as compression (I think). Since the ratio is used to + estimate the free space, this value is also different. + +- The external utility dutil displays garbage. + + You may have an old version of dutil somewhere in + your path, find and delete it and recompile dutil. If dutil displays just + zeros, you are definitely running a dutil from an older dmsdos version. + Remove it and recompile the source. + +- The external dmsdos utility displays some "lost clusters", but dos' + scandisk says everything is okay. + + The term "lost clusters" doesn't mean that the clusters are not assigned to + a file (like in a normal dos fs). Instead, it tells there are clusters that + have been allocated in the FAT but have a zero MDFAT entry. This is not a + filesystem error, it only means that the clusters do exist, but they don't + contain any data and therefore don't use disk space. So they aren't really + used, but they aren't free either. This is a special situation that can only + occur in a compressed filesystem. + +- I can't see long filenames on my win95 partitions, I only see the short + ones. + + Mount as type vfat instead of msdos. + +- I cannot unmount my dmsdos partition. umount complains with "device busy". + + Some process is currently using a file or a directory of your partition. + Find and kill it, then retry. This may be the external dmsdos daemon... + Hint: 'man fuser'. + +- I cannot read some directories in my CVF. They seem to be empty, but they + aren't empty under dos/win95. + + Watch for DMSDOS error messages in the kernel log. + +- On mount, I receive some strange error messages saying there were errors + in cluster xyz, but dos scandisk doesn't find them and even tells that the + CVF doesn't have such many clusters. + + Sorry, there *is* an error in your filesystem. However, it's a kind of + hidden error, i.e. it doesn't have an immediate effect. It's due to + garbage in some currently unused parts of the FAT or MDFAT. Most likely + the slack of the FAT or MDFAT is not zerod out. It should be zerod out + in order to represent still a valid FAT/MDFAT in case the compressed + filesystem is enlarged (this may even happen by increasing the compression + ratio). Dos scandisk & Co. don't check the unused FAT/MDFAT slack for + errors. I've been reported that these errors can be fixed by setting a + large compression ratio, running scandisk again (now it should find and + correct the errors), and then resetting the compression ratio. + +- The dmsdos daemon always exits with an error "Can't get permissions...." + + You are using the *internal* dmsdos daemon which is automatically started + when it is needed. In this case there's no need for the external daemon. + Watch your process list or look for a + '#define INTERNAL_DAEMON' in dmsdos-config.h. (The internal daemon is a + kernel process like kflushd or nfsiod.) See file dmsdos.doc for details. |
