STD-F100: File System

MMCC's PL/B notes

Notes home

Intro

History

Article index

MMCC Standards

v1.10

Mid-Michigan Computer Consultants, Inc.
509 Center
Bay City, Michigan
Sales (989) 892-9242 Support (989) 686-8860

ANSI Standard PL/B Language
MMCC Programming Standards

NOTE TO OUR READERS This web resource is written for our own use. But we feel strongly that the PL/B language should be shared with the software community. So feel free to use this at will! BUT PLEASE... if you find errors or omissions or have a better way to do something. TELL US! Dialog helps us all. Send e-mail to: support@mmcctech.com

FILE SYSTEM CONCEPTS

MMCC I/O and FILEs STANDARDS

MMCC software is written in a systematic way. There are several foundation concepts which, when understood, set the stage for all development. Central to everything is the "file system".

The MMCC file system is built on four code modules which are included into all programs. These are:

The file EQU list
xxxxIOFD.PLS - file descriptions
xxxxOPEN.PLS - file open / preps
    Filelist special case prep.
xxxxIO.pls - file read / write / update code
FileList I/O considerations

Sequential Read Loops for loading lists.

8802 - indexing utility program
8804 - general file testing program

Back to Standards Index
(Where xxxx is the prefix of the particular system. i.e. ADRW, SHOP, FM-, GHCW, ODA- etc.)

These modules contain the necessary code for every file. The common characteristics of the modules are:

Each file is identified with a two or three character prefix. A Work Order header file might be prefaced by "WH", a work order detail file by "WD". These characters are unique within the application system namespace and will not be violated.

Each file description and processing routine is encapsulated in a %IF compiler construction container. The %IF tests an equated variable named "xxIO" where "xx" is the file's unique identifier. The work order header equate would be WHIO.

Every program includes a list of every file's EQUATE. The program also included all three of the definition files. The value of the equate tells the compiler what parts of the includes should be part of the program as follows:

The file EQUATE list

Every program contains a list of EQU statements identifying all of the files in the system. The equated value for each file indicates how the file is to be used by the program. The greater the value, the more that can be done.

Besides controlling the compile proccess, this list provides quick documentation about how the program works. By simple inspection you know which files are used, which are just read, which are potentially updated.

The Equate values are:

xxIO EQU 0:
    Ignore the file and include nothing.
xxIO EQU 1:
    Include only the file description (FD).
xxIO EQU 2:
    Include the FD.
    Open in READ mode.
    Include READ, READKS and READKP routines.
xxIO EQU 3:
    Include the FD.
    Open in SHARED mode.
    Include READ, READKS and READKP, WRITE and UPDATE routines.
Equates for every file shoule be included in each program. If an equate is left out the compiler will give a warning and the equate will be assumed to be zero.

When a new file is added to the system it is not necessary to fix every program. Those which don't need the file will have the equivalent of a zero equate which says to ignore the file anyway.

An example of the equate list would be:
ASIO EQU 0 CAIO EQU 1 ;just need FD GXIO EQU 0 HCIO EQU 3 ;need full I/O MAIO EQU 0 ... etc ... RPTIO EQU 0 USERIO EQU 2 ;just need READs WDIO EQU 3 ;need full I/O WHIO EQU 0 XCIO EQU 3 ;need full I/O . INCLUDE xxx-IOFD

Back to Standards Index

xxx-IOFD.PLS - The File Descriptions

The "IOFD" include unit contains the "file descriptions". Every file is described in this one module. This module defines data fields and is typically included after the other variables and before the processing.

Each file's code is encapsulated in the %IF compiler control as follows:
%IF WHIO > 0 work order header file description %ENDIF
The FD contents have several consistent components.

The files are named "xxFILE" where "xx" is the file's identifier.

Each file has a field named xxNIF which is a one byte flag to indicate the status of the last I/O operation.

"NIF" is a throwback to an old system where it stood for "not in file". Today the flag has several values. Typically all we test for is "N", which means "no error found".

Each file will have a field named xxRECKEY. This will be a string to contain the record key. Some files may also have a xxSVKEY and some other related fields.

In the early days of PL/B, labels were limited ot 8 bytes. As a result, some files may have slightly varied forms of the xxRECKEY. A three byte file identifier is sometimes used and the record key string may be named xxxRECK.

All files are designed to have the primary key defined contigously starting from byte 1. This key will be encapsulated in a LIST structure named xxKEY.
WHKEY LIST WH_TYPE DIM 1 WH_WONUM DIM 8 WH_RECODE DIM 2 LISTEND
The xxRECKEY key string is used for all I/O. It can be packed up using
PACKKEY xxRECKEY from xxKEY

The BODY of the record follows the key and is encapsulated in a LIST structure named xxRECORD.

All I/O operations are directed to the LISTS, not the individual fields. This insures that the record format is never abused. Older PL/B programs would list every field in the I/O instruction. If the record changed, the instructions might not all be fixed. The beauty of the LIST is that the list exactly reflects the structure of the record. Adding a field to the LIST automatically communicates the change to all I/O instructions.
WHRECORD LIST WHUID DIM 7 WH_WONUM_B4 DIM 8 WH_WONUM_LINK DIM 8 ... etc... WH_DOT INIT "." LISTEND
A common practice has been to make the last field a single byte dot. The objective is to insure that indexed file records don't get truncated if written sequentially. This also makes a good target to look for when examining the raw text file to see if fields are properly defined and producing records with the correct length.

The xx_DOT field is initialized in the FD with the period character. The WRITE and UPDATE routines sould move a constant "." to the field to insure that it has the correct value.

A complete FD code set might be:
%IF WHIO > 0 .................................... . Work Order Header File . WHFILE IFILE BUFFER=450 WHRECKEY DIM 11 SV_WHRECKEY DIM 11 WHNIF DIM 1 . WHKEY LIST WH_TYPE DIM 1 WH_WONUM DIM 8 WH_RECODE DIM 2 LISTEND . WHRECORD LIST WHUID DIM 7 WH_WONUM_B4 DIM 8 WH_WONUM_LINK DIM 8 ... etc... WH_DOT INIT "." LISTEND %ENDIF

Back to Standards Index

xxx-OPEN.PLS - The File Opening routines

The "OPEN" unit contains the file opening and prepping code. This module is to be included IN-LINE near the beginning of the program. It is not a CALLED routine. The program just flows into the INCLUDE and out the other side.

EXCEPTION:

For "loadmods" (separately compiled modules), the xxx-OPEN routine might be put in a module to be called. The idea is that the mainline is entered oncce and does the opens during initialization. The "loadmod" may be called multiple times in the same session. The first call to the "loadmod" loads that program into memory. Once loaded it stays there and it's context is preserved. If files are opened on the first entry they don't have to be opened on subsequent entries. It's just a waste of time.

To make the most of this fact, we typically do use a CALL in the loadmod's initialization code. The routine that is called includes the xxx-OPEN routine and sets a flag to see if it needs to be processed:
#FILES_ARE_OPEN INIT "N" OPEN_THE_FILES IF (#FILES_ARE_OPEN = YES) RETURN ENDIF MOVE YES, #FILES_ARE_OPEN INCLUDE xxx-OPEN RETURN

Every file is coded in this one OPEN module. Each file's code is encapsulated in the %IF compiler control as follows:

xxIO EQU 2: Open in READ mode.
xxIO EQU 3: Open in SHARED mode.
The code for a typical open follows. Note that DISPLAYs are included but will only appear if the WINSHOW operation is done to turn on the main window. That's controlled by the PRIVDIAG flag which is set universally from the main menu.

The FILE PREP is part of the open routine. It is only available for files with read/write access. If a read-only file is not found the program errors out.

File's should NOT have to be prepped after the first time they're used. Most files are permanent. The PREP is included in this module because it's a logical, central place to have it. (Some older systems do not have the PREP code.)

Note that this is a nested %IF structure. The xxIO equate must be 2 or greater to get the module included at all. The first level sets the common characteristics, like the file name and displays the filename.

Note that the TITLE_LINE universal work area is packed up with each of the opens. The PACK instruction appends TITLE_LINE to itself, includes a "return" (HEX_7F), then the file name. On return to the mainline program, TITLE_LINE can be ignored or can be shown in an edit text or anywhere else.
%IF WHIO > 1 TRAP NOFILE IF IO PACK FILENAME, FILE_PATH,SYSTEMID,"-FWH" PACK TITLE_LINE,TITLE_LINE,HEX_7F: "WH_FILE: Work order header file:",FILENAME DISPLAY "WH_FILE: Work order header file": *H=40,*HON,*LL,FILENAME,*PL,*HOFF,*H=70; %IF WHIO < 3 DISPLAY "READ" OPEN WH_FILE, FILENAME,READ ADD ONE,OPEN$R %ELSE IF MOVE NO,FILENIF TRAP FILEERR IF IO ;IN I/O... just sets FILENIF DISPLAY "SHARE" OPEN WH_FILE,FILENAME IF (FILENIF=NO) ADD ONE,OPEN$S ELSE DISPLAY "WH file not found. Err:",S$ERROR$: " Name:",*LL,*HON,FILENAME,*HOFF,*PL ALERT PLAIN,"OK TO PREP WH FILE",ALERT_RESULT,"WHFILE" IF (ALERT_RESULT != 1) DISPLAY "Run aborting" STOP ENDIF PREP WH_FILE,FILENAME,FILENAME,"1-11","625",SHARE MOVE #ALL_ZEROS,WHRECKEY UNPACK NUL, WHRECORD CALL WHIOWRT ENDIF %ENDIF %ENDIF
The OPEN$R and the OPEN$S variables are simple counters for the number of files opened in read and the number opened in share mode. This was important in the DOS days when we had a limited number of file handles based on the FILES=nn value in CONFIG.SYS. It's not as important now, but can be interesting.

The TRAP NOFILE at the beginning of the routine is the general trap for a missing file. It's primarily used for read only files. If the trap is taken, the routine essentially aborts the program with an error message.

For read/write access files we use TRAP FILEERR IF IO. That routine is found in the xx-IO module. All it does it MOVE "X", FILENIF.

(FILENIF is a one byte field defined in the xx-IOFD module. It's used anywhere we need a temporary flag for testing file errors, like the open.)

If the file is not found on a read/write level open, we offer the user the oppertunity to prep the files. All of our files are permanent. We should get the prep offer only when new files are added to the system.

Most of our systems include a program which opens ALL files. After opening it shows the list of all the files. You can click on any file and get a display of the contents of the first record.

When a new file is added to the system, the 8804 program is compiled and run. It offers the PREP which we do. After that we should never see the PREP again.

Note that the PREP uses the form which describes the KEY position as "1-nn". That is significant. By doing that the system will later allow the SUNIDXNT utility with the "r" parameter to "reindex" the file. You could also prep with the key designated as "nn". The effect is the same as "1-nn" but you can't do the "r" utility index then.

SPECIAL CASE FOR PREPPING FILELIST and MULTIPLE INDEXES

Filelists are structures which put several indexes into a control group. If you update the primary file, the runtime will update all of the associated indexes.

Filelist present some challenges when prepping the files. If the text file and the several indexes already exist, you open the "primary" file first then open all of the secondary index files. If the files do not exist, each of the indexes must be created outside of the filelist group before the group can be opened.

Our solution to this problem is to put the primary file OPEN in a LOOP structure. If the primary file is found, the loop is broken and we open the secondary indexes. If the primary is NOT found, we prep all of the secondary files using a dummy file, then loop back and open the primary again. It should be found the second time and break the loop.

Note that each of the secondary index PREP's uses FILENAME to describe the TXT file but defines it's own #ISINAME for the ISI file name. The last prep is for the primary file and it uses the same FILENAME in BOTH of the name slots.

Once all of the files have been prepped, and the primary file has been successfully opened, we fall into the regular open for the secondary indexes. These trap a special NO_ISI_FILE routine in case the secondary index fails. We just want to be sure we know the type of file that caused such errors.

The code is as follows:
%IF ALIO > 1 TRAP NOFILE IF IO PACK FILENAME, FILE_PATH,SYSTEMID,"-AL00" PACK TITLE_LINE,TITLE_LINE,HEX_7F: "ALLIST: Master address list file:",FILENAME DISPLAY "ALLIST: Master address list file": *H=40,*HON,*LL,FILENAME,*PL,*HOFF,*H=70; %IF ALIO < 3 DISPLAY "READ" OPEN ALLIST, FILENAME,READ ADD ONE,OPEN$R %ELSEIF ALIO > 2 LOOP MOVE NO,FILENIF TRAP FILEERR IF IO DISPLAY "SHARE" OPEN ALLIST,FILENAME IF (FILENIF=NO) ADD ONE,OPEN$S BREAK ENDIF DISPLAY "AL file not found. Err:",S$ERROR$: " Name:",*LL,*HON,FILENAME,*HOFF,*PL ALERT PLAIN,"OK TO PREP WH FILE",ALERT_RESULT,"WHFILE" IF (ALERT_RESULT != 1) DISPLAY "Run aborting" STOP ENDIF . PACK #ISINAME, FILE_PATH,SYSTEMID,"-AA00" DISPLAY "Prep ",*HON,*LL,FILENAME,*PL,*HOFF: ", ", *HON,*LL,#ISINAME,*PL,*HOFF PREP #TEMPISI,FILENAME,#ISINAME: "U1-2,66-85,3-9","650",SHARE CLOSE #TEMPISI . PACK #ISINAME, FILE_PATH,SYSTEMID,"-AC00" DISPLAY "Prep ",*HON,*LL,FILENAME,*PL,*HOFF: ", ", *HON,*LL,#ISINAME,*PL,*HOFF PREP #TEMPISI,FILENAME,#ISINAME: "U66-85,1-9","650",SHARE CLOSE #TEMPISI . DISPLAY "Prep ",*HON,*LL,FILENAME,*PL,*HOFF: ", ", *HON,*LL,#ISINAME,*PL,*HOFF PREP #TEMPISI,FILENAME,FILENAME: "1-9","650",SHARE MOVE #ALL_ZEROS,ALRECKEY UNPACK ALRECKEY, ALKEY UNPACK NUL, ALRECORD WRITE #TEMPFILE, ALRECKEY;ALKEY,ALRECORD CLOSE #TEMPISI REPEAT %ENDIF %ENDIF . TRAP NO_ISI_FILE IF IO %IF ALIO > 1 PACK FILENAME, FILE_PATH,SYSTEMID,"-AA00" PACK TITLE_LINE,TITLE_LINE,HEX_7F: "AALIST: Master address list ALPHA IDX:",FILENAME DISPLAY "ALLIST: Master address list ALPHA IDX:": *H=40,*HON,*LL,FILENAME,*PL,*HOFF,*H=70; %IF ALIO < 3 DISPLAY "READ" OPEN AALIST, FILENAME,READ ADD ONE,OPEN$R %ELSEIF ALIO > 2 DISPLAY "SHARE" OPEN AALIST, FILENAME ADD ONE,OPEN$S %ENDIF %ENDIF . TRAP NO_ISI_FILE IF IO %IF ALIO > 1 PACK FILENAME, FILE_PATH,SYSTEMID,"-AC00" PACK TITLE_LINE,TITLE_LINE,HEX_7F: "ACLIST: Master non-class IDX:",FILENAME DISPLAY "ACLIST: Master non-class IDX:": *H=40,*HON,*LL,FILENAME,*PL,*HOFF,*H=70; %IF ALIO < 3 DISPLAY "READ" OPEN ACLIST, FILENAME,READ ADD ONE,OPEN$R %ELSEIF ALIO > 2 DISPLAY "SHARE" OPEN ACLIST, FILENAME ADD ONE,OPEN$S %ENDIF %ENDIF

Back to Standards Index

xxx-IO.PLS - The I/O (read / write) Routines

The "I/O" unit contains all "I/O" related code for the files. Each operation is designed as a called routine. This include module is usually included at the tail end of the mainline program.

Each file's code is encapsulated in the %IF compiler control just as all other file components described in this article.

This include unit may also contain a number of utility routines that all programs can take advantage of. These are not generally controlled by EQUATES. They're just in every program. Some are file specific and they may be included based on the EQUATES. Most of these utility routines appear at the beginning of the xxx-IO.PLS module.

Another large block of utility routines is in the MMCCSERV routine, which is stored in the SUNDB folder. The xxx-IO include module normally includes the MMCCSERV module as the last thing. That makes those routines available to all programs.

For the most part the I/O is done using the FILEIO verb. That means that a single, general format I/O statement can be written and it will be specialized at runtime based on a flag. We use the variable IOMETHOD, defined in COMMONWK to identify the method.

You don't have to worry about the I/O method in your program. That part is handled by the I/O routine itself. To read a record you'd typically do something like this:
MOVE SPACE to WH_TYPE MOVE INPUT_WO_NUMBER to WH_WONUM MOVE SPACE to WH_RECODE PACKKEY WHRECKEY from WHKEY CALL WHIOREAD IF (WHNIF != NO) { process error } ENDIF
Note that that errors are reported back in the xxNIF variable. If there are no errors the flag will be set to "N". The PL/B runtime will store the full error code in the S$ERROR$ string. You can display that or do whatever is needed if an error is detected.

All error conditions are trapped. The trap processing routines are T_xxxx which are common routines used by every I/O routine. The common trap routines put the flag into the TEMPNIF field which is then moved to the file's own xxNIF field after the operation.

Note the WRITE in the following example. The first thing it does is to UNPACK the xxRECKEY into the xxKEY LIST. That's because the I/O routine just names the LIST, not the individual fields. We want to be certain that the packed up key fields are actually in the fields which are written. That's just a peace of mind move since the caller should have insured that the fields are properly loaded as well.

The actual code in an xxx-IO.PLS module looks like this.

NOTE: This example is for standard, single index files. The routine is different for FILELIST file groups and is described after this example.
%IF WDIO > 1 ................................... . wdioread MOVE ONE,IOMETHOD GOTO #WDIODOIT . wdiorks MOVE FOUR,IOMETHOD GOTO #WDIODOIT . wdiorkp MOVE FIVE,IOMETHOD GOTO #WDIODOIT . %ENDIF %IF WDIO > 2 wdiowrt MOVE TWO, IOMETHOD UNPACK WDRECKEY,WDKEY MOVE ".", WD_DOT GOTO #WDIODOIT . wdioupd MOVE SIX, IOMETHOD MOVE ".", WD_DOT GOTO #WDIODOIT . wdiodel MOVE NO,TEMPNIF DELETE WD_FILE CALL T_OVER IF OVER MOVE TEMPNIF,WDNIF RETURN %ENDIF %IF WDIO > 1 #wdiodoit MOVE NO,TEMPNIF TRAP T_IOTRAP IF IO TRAP T_FTRAP IF FORMAT TRAP T_RTRAP IF RANGE FILEPI 1;WD_FILE FILEIO WD_FILE,WDRECKEY,IOMETHOD;WDKEY,WDRECORD CALL T_OVER IF OVER TRAPCLR IO TRAPCLR FORMAT TRAPCLR RANGE MOVE TEMPNIF,WDNIF IF (WDNIF=NO &: (IOMETHOD=4 | IOMETHOD=5)) PACKKEY WDRECKEY,WDKEY ENDIF IF (WDNIF=NO &: (IOMETHOD=3 | IOMETHOD=4 | IOMETHOD=5)) GOTO WHIOEDIT ENDIF RETURN . whioedit .... do any common editing work here ... RETURN %ENDIF

SPECIAL NOTES for FILELISTs

Where multiple indexes are needed the new PL/B FILELIST techniques are used. A file list is just a list of files with pre-defined keys. When a WRITE, UPDATE or DELETE is done to the file list, every file in the list is automatically updated. Before FILELIST was added, the programmer would have to treat each of the files separately and coordinate them to insure that the multiple indexes were updates.

Note that FILELIST are for output instructions only. The reads are done to the individual files. The PL/B runtime recognizes the read to a file that's part of a list and insures that all files are updated in subsequent outputs to the list.

Note also that the WRITE, UPDATE and DELETE operations must be done to the "primary" file. That file is the first one listed in the group and the first one opened.
%IF ALIO > 2 aliowrt MOVE TWO, IOMETHOD UNPACK ALRECKEY,ALKEY MOVE ".", AL_DOT GOTO #ALIO_FL_DOIT . alioupd MOVE SIX, IOMETHOD MOVE ".", AL_DOT GOTO #ALIO_FL_DOT . aliodel MOVE NO,TEMPNIF DELETE WD_FILE CALL T_OVER IF OVER MOVE TEMPNIF,WDNIF RETURN . #alio_fl_doit MOVE NO, TEMPNIF MOVE ".",AL_DOT TRAP T_IOTRAP IF IO TRAP T_FTRAP IF FORMAT TRAP T_RTRAP IF RANGE IF (IOMETHOD = 2) CLOCK TIMESTAMP,AL_ADDED WRITE AL_FILELIST;ALKEY,ALRECORD ELSE CLOCK TIMESTAMP,AL_CHANGED UPDATE AL_FILELIST;ALKEY,ALRECORD ENDIF CALL T_OVER IF OVER TRAPCLR IO TRAPCLR FORMAT TRAPCLR RANGE MOVE TEMPNIF,WDNIF RETURN %ENDIF

Back to Standards Index

Sequential Processing Loops

A common requirement for processing indexed files is to establish a starting point in the file then read forward sequentially. For example, say you want to look up the Smiths in an alpha index. You'd position the file to "SMITH", then read forward from that point.

In many cases you'll ask a user to give you a starting point. In the "SMITH" example they might enter "S" or "SMI" or "SMITH". You can set up the key and do a READ against the file. If the key is found the record will be read. If the key is not found, the file will be positioned to the closest record to the key and the next READKS (key sequential) will get the next record in line.

Since you won't know if the key the user enters will be a valid one or just a starting point, you have to do some tests around the read to see what the results are. We do this by taking advantage of our standard I/O routines and the way the xxNIF flag is set.

Here's how we do it:
MOVE USERS_KEY, AARECKEY MOVE ONE, AANIF LOOP IF (AANIF = "1") CALL AAIOREAD MATCH NO, AANIF CONTINUE IF NOT EQUAL ELSE CALL AAIORKS ENDIF MATCH NO,AANIF BREAK IF NOT EQUAL ...... proccess this record REPEAT
The AANIF variable is a string. When we move ONE, a one byte numeric variable with the value 1, to AANIF we'll get a string "1". (We could also move a literal.)

In the loop we test for the "1". That is a first time condition. If it's true we do the READ, which will reset AANIF to a "N" or some value other than "1". The second time through the loop AANIF will never be a "1" so we'll always do the READKS.

On the off chance that the user enters a valid key, that first READ will find the record. In that case AANIF will be "N". We'll fall out of the IF statment and process that record. Looping back, AANIF will not be a "1" so we'll do the READKS.

It's an elegant and simple routine.

Back to Standards Index

The 8802 general file indexing program

Each of the MMCC systems includes an xxx-8802 program. These are all about the same. Every file in the system is listed in a table along with it's keys and, if required, any secondary indexes.

At runtime, the 8802 program builds a series of check boxes and buttons on the screen; one for each file.

The user can put a check in each file that needs to be indexed. There is also a button to "Check All" and one to "Clear All".

Besides the check box, each file has a description on screen and a button out to the right for viewing the history of the last index. The button shows the two character file ID and the date that the index was last run. If you click that button you are shown the history file from that last index run.

There's a button to run the indexes for those files which are checked. The program builds a command line and executes the SUNIDXNT utility to index the files.

The SUNIDXNT command includes the "-l" argument telling it to save the screen output to a log file. After the index is run, that log file is read and stored in the "XC" file. (This was the original use of the XC file... it was called the "index control" file, thus XC.)

Another button titled "Build BAT File" will build a standard batch (.BAT) file to run all of the indexes. We usually build one of these so that if a problem occurs and we can't run the system, we can go to the command prompt and run the bat file to index the files. That file also provides a quick place to find the key specs for the files.

Back to Standards Index

The 8804 general file testing program

Most of the MMCC systems also include an "xxx-8804" program. This one is written to open every file using the standard xxx-OPEN include unit. If the file is missing you get the opportunity to prep it.

When the files are open, the program shows a list of all file, their names, and descriptions. You can click on any file and get an analysis of the first record of the file. This is handy for testing a new file to be sure that the design record length agrees with the file description.

When a new file is added to the system, the only change needed to the 8804 program is to add the one equate in the EQU list. When the program is compiled everything comes in via the file include units.

This is our normal testing routine for new files. Add them to the 8804 and compile it. Run it to prep the file. Then do the first record analysis to insure that the file is OK. If there's a problem, delete the file, fix the FD, compile and test again.

Note that the old DOS systems used an "8804" program for setting standard screen colors. We're not sure why we didn't catch that when we began to write Windows systems. There's no need for the color setting program in Windows so the name conflict was just missed.

Back to Standards Index

© 2001 mmcc All Rights Reserved.

Report problems or suggestions to support@mmcctech.com