Clipper: (Last update: 12/25/2000)

Databases (.DBF) - Memo files (.DBT) - Index files (.NTX) - Mem files (.MEM)

0. Explanations:

1. Structure of Clipper databases
(annotations to other systems as known):

1.1. Header (32 bytes):

Name Byte Position Length Type Content
Kind 0 1 1 B
Kind Usage
03H Clipper or dBase without Memo field/file
83H same with Memo field/file
07H Visual Objects without Memo
87H same with Memo
30H Visual Foxpro without Memo
F5H same mit Memo
Year 1 2 1 B Year of last change, only two digits!
Month 2 3 1 B Month of last change
Day 3 4 1 B Day of last change
Count 4 5 4 DW Record count
Start 8 9 2 W Address of start of data
(addresses start at 0!)
RecLength 10 11 2 W Record length (incl. 1 byte, "deleted flag")
  12 13 20 B Reserved, unimportant, always 00H?

1.2. Field entries (32 bytes), one entry per existing field:

Name Byte Position Length Type Content
Fieldname 0 1 11 B Field name, ends with 00H,
maybe trailed by some smut
Fieldtype 11 12 1 B

Type

Meaning

C

Character

D

Datr

L

Logical

M

Memofield

N

Numerical

Foxpro etc. define additional types!
Fieldposition 12 13 4 DW Field position in record, unused by Clipper
but probably used by Foxpro
Fieldlength 16 17 1 B Field length
Fielddec 17 18 1 B Number of digits after decimal point
  Clipper uses for field type 'C' Fieldlength and Fielddec together as
type 'W' as field length for fields bigger than 255 chars.
Effective length: Fielddec * 256 + Fieldlength
  18 19 14 Byte Unused by Clipper

1.3. End of header:

1.4. Record:

The first byte contains always a blank (" ", ASCII 32, 020H)
for undeleted records or an asterisk ("*", ASCII 42, 02AH) for deleted ones.
This byte is always included in the Fieldlength word (Byte 10 and 11)!
The field contents follow. There is no separation character between the fields!

Field contents:

Type

Usage

C

Character, right padded with spaces if necessary

D

Date, always 8 bytes, saved as YYYYMMDD

L

Logical, 1 byte, saved as "T" or "F"

M

Memo, 10 byte pointer to the first 512 byte block in the DBT file

N

Numerical, saved with decimal point!

1.5. End of data: one byte, CTRL-Z = ASCII 26 = 1AH.
Remark This byte may or may not be there!

1.6. Programming example in Clipper

1.7. Programming example in C

2. Structure of Clipper memo files (.DBT):

2.1. Header (512 bytes):

Name Byte Position Length Type Content
Countl 0 1 4 DW Next new block number = count
of used 512 byte blocks
(last block may be only partially filled,
therefore the file size may not be a multiple of 512!)
  4 5 508 Bytes don't mind, reserved, may contain smut

2.2. Content:

You get the address of the data for a certain record in the DBF file by multiplying the above mentioned
10 byte pointer by 512. There you'll find the data. They're valid up to (but excluding) the first
CTRL-Z (ASCII 26 = 01AH).

Maximum size data for one record = 64 KB = 128 blocks with 512 bytes each!
Maximum size DBT file = 32 MB!

Remark: If the memofield of a record is empty there will be reserved 512 bytes anyway!
Remark: If the memofield gets bigger beyond the end of the block the data will be copied to the end of the DBT file (if it's not already the last block). The old blocks won't be used anymore. This space is lost until you do a PACK or a COPY.
Of course you regain the space too with a ZAP <vbg>.

3. Structure of Clipper index files (.NTX):

Clipper index files use a modified B+ tree algorithm.
Usually only the leafs have key values.
With Clipper the root and the nodes also have key values.

An NTX file has exactly one header block and contains data in at least one and at max three levels
depending on the following number: record count times (keylength + 8).
All data will be saved in blocks of 1024 bytes.
There is exactly one header block and exactly one block for level 1.
Blocks for level 2 and 3 exist only when needed. Then there may be lots of them.

3.1. Header (1024 bytes):

Name Byte Position Length Type Content
Type 0 1 2 W
Type Usage
0600H "normal" indexfile, ascending
0700H descending
0900H rumour has it that this means
a "good" indexfile
but I doubt it
Version 2 3 2 W Some people say that this field contains
the Clipper version. That can't be because
I found at least 20 different values in my
databases (all Clipper 5.2e).
FirstPage 4 5 4 DW Address of page level 1
FreePage 8 9 4 DW Start of chain of free pages (3.4.)
if there are any otherwise zero
Entrylength 12 13 2 W Length of entry = key length + 8
(4 bytes address next indexblock + 4 bytes
recordnumber in DBF file, see below)
Keylength 14 15 2 W Length of key not of key expression!
Decimaldigits 16 17 2 W Decimal digits if there are any in the index
MaxEntries 18 19 2 W Maximum number of index entries per index page
MinEntries 20 21 2 W Minimum number of index entries per index page,
needed to balance the index tree
Key 22 23 257 B keyexpression delimited with 00H, after that smut
Unique 279 280 1 B Unique index = 01H, 00H otherwise
  4 5 744 B Unimportant, reserved, may contain smut

3.2. Level 1-3 (1024 bytes):

Name Byte Position Length Type Content
Count 0 1 2 W Count of valid entries
Important: if the level is not the lowest one used
you have to add 1. Why? Because!
Entrypointer 2 3 2 W n * entrypointer, n = MaxEntries (see last table)
Remark: maybe a lot of these entries contain data,
only the first n are valid!
The pointer is meant to be inside this block only!
Indexentries         n indexentries, n = MaxEntries (see 3.1.)
structure indexentry (see 3.3.)
Remark: maybe a lot of these entries contain data,
only those are valid that are pointed to by a pointer!

3.3. Indexentry (8 bytes + keylength):

Name Byte Position Length Type Content
Address 0 1 4 DW Address = 0 if lowest level
otherwise the address of the block of the next level
Recordnumber 4 5 4 DW If # 0 you'll find the record with the key (next field)
in the DBF file under this recordnumber
Key 8 9     Key with length (3.1.) without 00H!

The key of an index entry (if not lowest level!) is truly greater than all the keys
of the subordinate block whose address is in the same indexentry.

The last index entry of level 1 has an empty key, 00H.
In the according subordinate block are the keys with the highest values!

3.4. Free blocks:

According to theory free blocks (after deleting or so) will be chained and eventually reused.
The block number of the first free block is in the header (see 3.1.).
I personally never have seen that Clipper really has free blocks.
But maybe I'm packing my databases too frequently?!

3.5. Access to the data in the DBF file according to the record number found in the NTX file:

You get the address of the record in the DBF file you may compute like this:
(recordnumber - 1) * record length (s. 1.1.) + Start (s. 1.1.)
Remember: the first byte is the deleted flag!

3.6. Example for an index file (all values hexadecimal):

Header address 1. page: 00 00 0C 00
keylength: 3
maximum entries per page: 4C
minimum entries: 26
keyexpression: STR(ART_NR,3)
unique: True
level 0 address: 00 00 0C 00
count: 03 (add 1 because it's not the lowest level!)
entry pointer 1:
00 BD
entry pointer 2: 00 9C
entry pointer 3: 00 B2
entry pointer 4: 00 A7
address 00 BD:
= address
00 00 04 00
= record number 00 00 00 B1
= key '256'
address 00 9C:
= address 00 00 14 00
= record number 00 00 00 35
= key '517'
address 00 B2:
= address 00 00 08 00
= record number 00 00 00 05
= key '872'
address 00 A7:
= address
00 00 10 00
= record number 00 00 00 00
= key 00H
level 1 address: 00 00 04 00
count: 26H(1 addieren!)
entry pointer 1:
00 F4
...
entry pointer 27H: 00 E9
address 00 F4:
= address
00 00 24 00
= record number 00 00 01 99
= key '100'
address 00 E9:
= address 00 00 08 20
= record number 00 00 03 A5
= key '254'

...

address: 00 00 10 00
count: 31H (1 addieren!)
entry pointer 1: 00 E9
...
entry pointer 32H: 01 A0
address 00 E9:
= address 00 00 40 00
= record number 00 00 02 07
= key '880'
address 01 A0:
= address 00 00 06 24
= record number 00 00 04 01
= key '990'
level 2 address: 00 00 24 00
count: 36H
entry pointer 1:
00 C8
...
entry pointer 36H: 01 20
address 00 C8:
= address 00 00 00 00
(lowest level!)
= record number 00 00 01 99
= key ' 10'
address 01 20:
= address 00 00 00 00
(lowest level!)
= record number 00 00 03 55
= key ' 99'
... address: 00 00 08 20
count: 28H
entry pointer 1:
00 D3
...
entry pointer 28H: 00 FF
address 00 D3:
= address 00 00 00 00
(lowest level!)
= record number 00 00 03 77
= key '154'
address 00 FF:
= address 00 00 00 00
(lowest level!)
= record number 00 00 03 05
= key '250'
... address: 00 00 06 24
count: 38H
entry pointer 1:
00 DE
...
entry pointer 38H: 02 33
address 00 DE:
= address 00 00 00 00
(lowest level!)
= record number 00 00 01 88
= key '881'
address 02 33:
= address 00 00 00 00
(lowest level!)
= record number 00 00 03 C5
= key '979'

4. Structure of Clipper mem files (.MEM):

For each and every variable that has been saved with "SAVE TO" there is a header (4.1.), that's really similar to the field entries of the DBF file (s. 1.2). Surprise, surprise!
After the header there are the data (variable length depending on the data type) as shown in (4.2.).
End of informations s. 4.3.

4.1. Header for variables:

Name Byte Position Length Type Content
NameOfVariable 0 1 11 B Name of variable delimited with 00H,
probably followed by some smut
TypeOfVariable 11 12 1 B  The high bit is set here. So you have to subtract 128
from this value to get the type

Type

Meaning

C

Character

D

Date

L

Logical

N

Numerical

  12 13 4 DW don't care, reserved
LengthOfVariable 16 17 1 B Length of variable
DecOfVariable 17 18 1 B Decimal digits
  For type of variable 'C' LengthOfVariable and DecOfVariable
will be taken as word for strings greater than 255 bytes.
Effectiv length: DecOfVariable * 256 + LengthOfVariable
  18 19 14 B don't care, reserved, unused?

4.2. Content of variables, data:

Type

Usage

C

Character, as many digits as stated in the length field, the trailing byte 00H
is included.

D

Date, always 8 bytes, saved as number of day in IEEE format, not so easily readable,
after the conversion you have to subtract 2415021 to get the number of days between
this date and the 01/01/1900.

L

Logical, 1 byte, 01H = .true., 00H = .false.

N

Number, always 8 bytes, saved in IEEE format,
length and decimal digits as in table 4.1

4.3. Last byte: CTRL-Z = ASCII 26 = 01AH

Program to dump a Mem file

programming page

starting page

(www.zelczak.com/clipp_en.htm)