Winning at MPE

WINNING AT MPE by Eugene Volokh, VESOFT Q&A column published in INTERACT Magazine, 1983-1986. Published in "Thoughts & Discourses on HP3000 Software", 1st-3rd ed. Q: I work in a college environment, where we sometimes do a backup during the day. When the backup is in process, not only can't I write to some files, but I also can't run my programs -- I get FSERR 40 (OPERATION INCONSISTENT WITH ACCESS TYPE). What can I do about this? A: Whenever a system backup (or any kind of SYSDUMP or :STORE) is in progress, all the files being backed up are locked to prevent write access until the file is actually written to tape. It turns out that despite the fact that program loading appears to be a read-only operation, the MPE loader actually tries to modify the program being loaded; the fact that it is being dump prevents the loader from doing this, and a load failure occurs. What can be done about this? For one, you could list the fileset containing the files that you think you'd want to use as the first fileset in the SYSDUMP fileset list; for instance, if you think you'll need to run programs in PUB.SYS, you can type "@.PUB.SYS, @.@.@" instead of "@.@.@" when prompted for the filesets to be dumped. Since files are only locked from the time the SYSDUMP begins to the time the file is actually dumped to tape, the files in PUB.SYS will be locked for only a short time because they'll be among the first to be dumped. IMPORTANT NOTE: Starting with the Q-MIT, the SYSDUMP and :STORE commands do not unlock the file until they finish writing the entire reel on which the file is stored. If you want them to unlock as they are dumped rather than wait until the reel is finished, you should follow the filesets to be dumped with ";ONERR=QUIT". Note that this also turns off tape error recovery. Another way to solve this problem is to :ALLOCATE the programs that you want to run. When a program is :ALLOCATED, it is loaded and remains loaded until it is :DEALLOCATEd; thus, there is no longer any reason for the loader to try to modify it, and it can thus be run even during a SYSDUMP, whether or not it has already been dumped. Note that even though a program is :ALLOCATEd, it will be backed up. It is thus a good idea to allocate all the compilers you use, SEGPROC.PUB.SYS (which executes the :PREP command), your favorite text editor, and some frequently used subsystems such as FCOPY, SPOOK, etc. Finally, you may find that you want to run a program which you have not :ALLOCATEd and which is still locked down, waiting to be dumped. Well, even though you can't run the file, you can always read it and thus copy it. If you had the foresight To :ALLOCATE FCOPY.PUB.SYS, just :FCOPY the program file into some new file (which will not be affected by the SYSDUMP because it did not exist at the time the SYSDUMP started), and run the new file. A running SYSDUMP does not mean that all work on your system must come to a grinding halt. There are several ways in which you can continue to do productive work (like playing MANSION) on your system even while a SYSDUMP is in progress. Q: I am a PASCAL/3000 user, and like its powerful, easy-to-use control and data structure. However, I have found that some of my PASCAL programs run a bit on the slow side. Is there anything I can do to speed them up? A: If you access your files as TEXT files, a few minor modifications can speed up your programs dramatically and also greatly decrease their impact on the entire system. Consider a PASCAL program that must sequentially read a fixed record length file with record length 80 bytes, blocking factor 16, and about 1400 records. There are two ways in which you can read this file: One is to declare the file as "VAR F: TEXT" and READLN from the file into a PACKED ARRAY [1..80] OF CHAR; i.e. VAR F: TEXT; R: PACKED ARRAY [1..80] OF CHAR; ... READLN (F, R); Another way is to declare the file as "VAR F: FILE OF PACKED ARRAY [1..80] OF CHAR" and READ form the file into a PACKED ARRAY [1..80] OF CHAR; i.e. VAR F: FILE OF PACKED ARRAY [1..80] OF CHAR; R: PACKED ARRAY [1..80] OF CHAR; ... READ (F, R); If all you need to do is read entire records (as opposed to reading strings or numbers) from the file, the above two methods are functionally identical. However, method number one, which uses a TEXT file, uses 87.4 wall seconds and 87 CPU seconds (tested on a lightly loaded Series 44) to run -- an average of 62 msecs per read; method number two, which uses a FILE OF PACKED ARRAY [1..80] OF CHAR, uses 4 wall seconds and 3.7 CPU seconds -- an average 2.8 msecs per read, a 20-fold improvement! Thus, IF YOU ARE SEQUENTIALLY READING LARGE FILES RECORD-BY-RECORD IN PASCAL/3000, DECLARE THEM AS FILES OF PACKED ARRAY OF CHAR, NOT AS TEXT FILES. Incidentally, one might think that reads of FILEs OF PACKED ARRAY OF CHAR use some kind of special method to work as fast as they do. This is not true -- a call to the FREAD intrinsic against the above file will also take about 3 msecs. Apparently, reads against TEXT files are just grossly, and probably unnecessarily, inefficient. I have not the foggiest notion what it is that takes 60 milliseconds per READLN for the PASCAL run-time library to do, but its author will have some explaining to do to St. Peter when he goes up yonder... Q: We have a program that is run by many users that, among other things, prompts the user for a password. To make it easier to change the password, we would like to keep the password in a disc file. However, if we restrict read access on the file to only the System Manager, our program will not be able to read the file when run by an ordinary user; on the other hand, if we allow read access to everyone, our program will always be able to read the file, but so would the users! How can we restrict access to the file to a particular program, not a particular user? A: One of the major failings of the MPE file security system is that you can not restrict access to a file by program, but only by user. However, like many MPE failings, this one can be gotten around in a number of ingenious ways. One method is the so-called "knowledge security" approach. Instead of restricting access to a file by who a user is, we restrict access to the file by what the user knows. In simpler terms, we release the file for public access but put a lockword on it; thus, only those people or programs that know the lockword can access the file. Then you hardcode the lockword in the program, and the program will be able to access the file but an ordinary user can not unless he knows the lockword. Another way to implement this "knowledge security" is by putting the password in an IMAGE database and putting an IMAGE password on the IMAGE database. Again, only a person or a program that knows the IMAGE password will thus be able to access the database and find the user password. The only flaw in the above system is that the very reason why you make the password easily changeable is that passwords very quickly "leak out". For instance, anyone who has READ access to the program file can dump it to the line printer and see the password in the object code. Although this hole can be plugged by keeping the password in an encoded form in the program or allowing users only EXECUTE access to the program, the fact remains: passwords do not remain secret for any long period of the time. The alternative method involves a little-known feature of the HP file security system. If, while in PRIVILEGED NODE (Egad!), a program tries to open a file with aoptions bits (12:4) set to 15 (all "one" bits), the file is opened for read access AND ALL SECURITY CHECKING AGAINST THE FILE (not including lockword checking) IS WAIVED. Thus, the file can be protected against access by anybody, but the program can read it by entering privileged mode and opening the file with aoptions (12:4) set to 15. Admittedly, at first glance this approach is inferior to the first method in that it uses a privileged mode, a fickle thing at best. However, this is a variety of privileged mode that is very unlikely to crash the system (as opposed to "HARD" privileged mode, like ATTACHIO or GETSIR, which should be used with much more caution -- see "Privileged Mode: Use and Abuse" by yours truly in the Sep/Oct 1982 issue of INTERACT Magazine (also in this EBOOK.VECSL group of files). Furthermore, it does not require a second password to protect the location of the main password, as the first method does. Nonetheless, since HP does not document this method and does not guarantee that it will not change in later versions of MPE, it should be used carefully and at your own risk. Good luck. Q: I am a KSAM user, and have recently come across a very bizarre problem while using KSAMUTIL. My conversation with KSAMUTIL went something like this: :RUN KSAMUTIL.PUB.SYS >PURGE DATAFILE NONEXISTENT PERMANENT FILE (FSERR 52) >BUILD DATAFILE;KEYFILE=KEYFILE;KEY=BYTE,1.8 DUPLICATE PERMANENT FILE NAME (FSERR 100) >EXIT How can my file both not exist and exist at the same time? A: You have fallen victim to the way HP stores KSAM files. KSAM file is actually two files: a data file and a key file, either of which may be purged with MPE's :PURGE without purging the other. What probably happened is that DATAFILE's key file was purged, but DATAFILE itself was not. When you tried to do a ">PURGE DATAFILE", KSAM tried to open the DATAFILE-KEYFILE pair, found that KEYFILE did not exist, and returned a NONEXISTENT PERMANENT FILE error; then when you tried to do a ">BUILD DATAFILE;KEYFILE=KEYFILE", KSAM tried to build the DATAFILE-KEYFILE pair, found that DATAFILE already existed, and returned a DUPLICATE PERMANENT FILE error. However, there exists a workaround for this problem. Instead of doing the above, do :PURGE DATAFILE :PURGE KEYFILE :RUN KSAMUTIL.PUB.SYS >BUILD DATAFILE;KEYFILE=KEYFILE;KEY=BYTE,1,6 >EXIT MPE's :PURGEs do not care whether or not the file being purged is a KSAM file; they just purge the file. That way, by the time you get to the BUILD, you will be guaranteed that neither DATAFILE nor KEYFILE exist. Q: Unlike some other languages, such as COBOL, PASCAL, and SPL, FORTRAN variables do not have to be explicitly declared; if you use an undeclared variable, FORTRAN automatically assumes a type for it. However, this feature is more often a hindrance than a help, because it makes it very easy for incorrectly-spelled variable names to go undetected. How can I make FORTRAN detect usage of undeclared variables? A: The problem that you mentioned is indeed a major problem; I once heard that a Mariner space probe missed Venus by a couple of million miles because a variable name was misspelled in the FORTRAN guidance program. Fortunately, there is a way to get around this problem. FORTRAN has a statement called the IMPLICIT statement, which redefines FORTRAN's implicit typing strategy; you can say something like IMPLICIT INTEGER (A-P,S-Y), REAL (R), DOUBLE PRECISION (Z) and it will tell FORTRAN to assume that all undeclared variables that start with R are REALs, all that start with Z are DOUBLE PRECISIONs, and all others are INTEGERs. Furthermore, FORTRAN has a COMPLEX type, used for storing complex numbers, which, needless to say, rarely find their way into business applications. Thus, if you say IMPLICIT COMPLEX (A-Z) all undeclared variables will be assumed to be COMPLEX. Furthermore, if you include a $CONTROL MAP at the beginning of your program, a variable reference map will be generated as part of your listing; and, if you redirect the listing to a disc file, you can then /TEXT the listing file and search through it for the string "COMPLEX"; whenever you see it, you know that it's in a reference map entry for an undeclared variable. An additional benefit of this approach is that COMPLEX variables are not compatible with many other types, and will often give outright errors if used inadvertently. Thus, the solution to your problem is: 1. Include a "$CONTROL MAP" and an "IMPLICIT COMPLEX (A-Z)" at the beginning of your program. 2. Compile the program with a listing redirected to a disc file. 3. Use your favorite text editor to scan the listing file for the string "COMPLEX", which should appear in reference map entries of undeclared variables. This is especially easy in Robelle's QEDIT, in which you can scan an external file for a string without leaving the file currently being edited. Q: I am writing a system program in SPL, and I'm sick and tired of having to do a MOVE BUFFER:="PLEASE ENTER YOUR NAME: "; PRINT (BUFFER, -23, 0); every time I want to print a message to the terminal; I find it especially worrisome to have to count the characters in each message I want to output (did you notice that the number of characters above is wrong?). Is there some better way? A: Yes. Using some ingenious DEFINEs, you can make you life a whole lot easier. Consider, for instance, the following: ARRAY OUT'BUFFER'L(0:127); BYTE ARRAY OUT'BUFFER(*)=OUT'BUFFER'L; BUTE POINTER OUT'PTR; DEFINE SAY = BEGIN MOVE OUT'BUFFER:= #, TO TERMINAL = ,2; @OUT'PTR:=TOS; PRINT (OUT'BUFFER'L, -(LOGICAL(@OUT'PTR)-LOGICAL(@OUT'BUFFER)), 0); END #; If you include the above declaration and DEFINEs into your code, you will be able to say SAY "PLEASE ENTER YOUR NAME: " TO'TERMINAL; and have "PLEASE ENTER YOUR NAME: " printed on the terminal. Easy as pie. How does this work? Well, the above statement translates out into BEGIN MOVE OUT'BUFFER:="PLEASE ENTER YOUR NAME: ",2; @OUT'PTR:=TOS; PRINT (OUT'BUFFER'L, -(LOGICAL(@OUT'PTR) -LOGICAL(@OUT'BUFFER)), 0); END; The ",2" on the MOVE tells SPL compiler to generate code that will leave the destination address, i.e. the address of the byte immediately after the last byte moved to OUT'BUFFER, on the stack. The OUT'PTR pointer is then made to point to this address; finally,when we subtract LOGICAL(@OUT'BUFFER) from LOGICAL(@OUT'PTR) we get the actual number of characters moved, and thus the length of the string to be output. We the use this length in the PRINT intrinsic call, outputting exactly as many characters as we moved. Note that we convert the addresses to LOGICAL in order to avoid certain little-known but very hazardous problems with byte address arithmetic, which are discussed in great detail in Robelle Consultants' SMUG II Microproceedings (call Robelle for more information). Also note that we do a BEGIN in the SAY define and an END in the TO'TERMINAL define; that way, IF I=J THEN SAY "EQUAL" TO'TERMINAL; will expand into IF I=J THEN BEGIN MOVE OUT'BUFFER:="EQUAL",2; @OUT'PTR:=TOS; PRINT (OUT'BUFFER'L,-(LOGICAL(@OUT'PTR) -LOGICAL(@OUT'BUFFER)),0); END; not IF I=J THEN MOVE OUT'BUFFER:="EQUAL",2; @OUT'PTR:=TOS; PRINT (OUT'BUFFER'L,-(LOGICAL(@OUT'PTR) -LOGICAL(@OUT'BUFFER)),0); Another advantage of these defines is that you can say BYTE ARRAY MY'ARRAY(0:24); ... SAY MY'ARRAY,(25) TO'TERMINAL; and have MY'ARRAY, a byte array, be output to the terminal without having to equivalence it with a logical array (which is what PRINT requires). Of course, other defines may be similarly created, such as NO'CR'LF define, which is just like the TO'TERMINAL define except that it does a PRINT with carriage control code %320 (no carriage return/line feed) instead of 0. Also, think a bit about this define: DEFINE XX= ,2; MOVE *:= #; What very useful function does it implement? The answer to this and more information on making life with SPL easier in the next column; tune in next month, same BAT-TIME, same BAT-CHANNEL... Q: How do we know that computer programming is the world's oldest profession? A: The Bible says that before the creation, there was chaos; how can you have chaos without a computer programmer? Q: I have a program that appends records to a file. I noticed that when I run it on one terminal and do a :LISTF of the file on another terminal, the :LISTF shows that the file's EOF is 65 records (what it was before I ran the program), even though my program has already appended 10 records. When the program terminates, the EOF reflects the number of records that have been appended to the file; however, I'd like to be able to see the real EOF when I do a :LISTF. How can I do this? A: The reason why the :LISTF of the file does not reflect the real EOF is that the :LISTF prints the contents of the file's "file label", a disc sector containing various information on the file such as the file's EOF. However, for performance consideration, the file label is not updated until the file is closed or a new file extent is allocated; if the file system were to update the file label each time a record is written, it would have to do twice as many I/Os as it otherwise would (one I/O to write the record and one I/O to update the file label). However, there are deeper problems with the way the file system does this than just the fact that a :LISTF does not always show correct data for open files. If the system crashes while the program is running, the information on the real EOF (which is stored in memory while the file is open) will be lost, and with it all the records that have been written since the EOF was last updated! Thus, if you have a file with FLIMIT 10,000 and 8 extents, up to 1,250 records (1 extents' worth) may be lost if the system crashes while the file is being appended to. Therefore, in many cases, you may find the deferred EOF posting that the file system uses more of a disadvantage that an advantage. Fortunately, you can explicitly command the file system to post the EOF to disc by using the FCONTROL intrinsic with mode=6. Thus, in COBOL instead of simply saying WRITE OUT-RECORD. you would say WRITE OUT-RECORD. CALL INTRINSIC "FCONTROL" USING OUT-FILE, 6, DUMMY. That way, the EOF will be posted to disc after the record is written and a system failure will cause the loss of at most one record. In addition to this, a :LISTF of the file will reflect the true EOF of the file. Q: I have found the option that allows redirecting :LISTF output very useful. Unfortunately, many other MPE commands, like :SHOWJOB and :SHOWOUT do not have this option. Is there any way to redirect :SHOWJOB or :SHOWOUT output to a disc file or the line printer? A: Yes, there is. First of all, the :SHOWJOB command actually does have this option, although it is not documented anywhere. Just do a :FILE LFILE,NEW;DEV=DISC;SAVE;NOCCTL;REC=-80,,F,ASCII :SHOWJOB JOB=@j;*LFILE Specifying a";*file" parameter will cause the :SHOWJOB output to be redirected to another file just as it would on the :LISTF command. However, other commands, like :SHOWOUT or :SHOWME, do not have this parameter, documented or undocumented. Fortunately, there is a trick that you can use to redirect their output to a file. The reason why some commands' output is not easily redirectable is that they write it to $STDLIST, not to a user-specifiable file (like :LISTF or :SHOWJOB) or to a formal file designator that can be redirected (like SYSLIST in :STORE or :RESTORE). But, even though you can't redirect an MPE command's $STDLIST, you CAN redirect a program's $STDLIST, and with it, the output of all MPE commands that it invokes. So, to redirect :SHOWOUT's output, just do the following: :FILE OUTFILE,NEW;DEV=DISC;SAVE;NOCCTL;REC=-80,,F,ASCII :RUN FCOPY.PUB.SYS;STDLIST=*OUTFILE;INFO=":SHOWOUT SP" What does this do? Well, it runs FCOPY with $STDLIST redirected to OUTFILE. Furthermore, it instructs FCOPY to execute a :SHOWOUT SP command. :SHOWOUT SP will then output the data to $STDLIST, which has been redirected to OUTFILE! So, OUTFILE will now contain the FCOPY output (its identification banner) and the :SHOWOUT output. Naturally, you can do the same from your program by calling the CREATEPROCESS intrinsic (see the Intrinsics Manual). Q: Tell me more about making life with SPL easier... A: First, some unfinished business. Last month, I introduced the "mystery define" DEFINE XX = ,2; MOVE *:= *; What this define does is allows you to concatenate two BYTE ARRAYs; doing a MOVE FOO:="Hello there, " XX NAME,(8) XX "!"; will move into FOO the string "Hello there," followed by the first 8 characters of NAME followed by a "!". Of course, the define can be used in the SAY ... TO'TERMINAL defines I mentioned last issue. Since XX is not a particularly good name for it, from now I'll call it CONCAT (for CONCATenation). For the benefit of those who missed last month's column, let me recap the DEFINEs that I talked about: ARRAY OUT'BUFFER'L(0:127); BYTE ARRAY OUT'BUFFER(*)=OUT'BUFFER'L; BYTE POINTER OUT'PTR; DEFINE CONCAT = ,2; MOVE *:= #; DEFINE SAY = BEGIN MOVE OUT'BUFFER:= #, TO'TERMINAL = ,2; @OUT'PTR:=TOS; PRINT (OUT'BUFFER'L, -(LOGICAL(@OUT'PTR)-LOGICAL(@OUT'BUFFER)), 0); END #, NO'CR'LF = ,2; @OUT'PTR:=TOS; PRINT (OUT'BUFFER'L, -(LOGICAL(@OUT'PTR)-LOGICAL(@(OUT'BUFFER)), %320); END #; Briefly, these let you do things like SAY "Error: Bad filename" TO'TERMINAL; << print "Error: Bad filename" to the terminal ... >> or SAY "Enter your name: " NO'CR'LF; << print "Enter your name:" without carriage return/line feed >> or SAY "Hello there, " CONCAT NAME,(8) CONCAT "!" TO'TERMINAL; instead of going through a lot of trouble moving data to byte arrays, counting the length of the array, and printing a logical array equivalenced to the byte array. However, all of the above only work on byte arrays and character strings. An equally needed feature is one that permits you to write numbers in a user-readable format, without having to call ASCII and PRINT explicitly. How is this to be done? The following declarations and DEFINEs accomplish our task: INTRINSIC ASCII; BYTE ARRAY ASCII'BUFFER(0:255); DEFINE ASCII'OF = ASCII'BUFFER,(ASCII #, FREE = ,10,ASCII'BUFFER) #; Now, to output, say, I to the terminal, just do a: SAY ASCII'OF (I FREE) TO'TERMINAL; Or, to output "The value of I is " followed by I followed by a ";", do a: SAY "The value of I is " CONCAT ASCII'OF (I FREE) CONCAT ";" TO'TERMINAL; Whew! How does this work? Well, let us consider what the first SAY command expands into: SAY ASCII'OF (I FREE) TO'TERMINAL; | SAY ASCII'OF (I,10,ASCII'BUFFER) TO'TERMINAL; | SAY ASCII'BUFFER,(ASCII (I,10,ASCII'BUFFER)) TO'TERMINAL; << we could go on further by expanding SAY and TO'TERMINAL, but we already know what they do >> Now, "ASCII (I,10,ASCII'BUFFER)" is a call to the system intrinsic ASCII, which, as you may know, will convert I into its base 10 representation which it will put into ASCII'BUFFER, and will return the length of the representation. so, the above expanded statement means the same as temporary:=ASCII (I,10,ASCII'BUFFER); SAY ASCII'BUFFER,(temporary) TO'TERMINAL; The first statement will perform the conversion; the second line is just an ordinary SAY ... TO'TERMINAL that we already know about, which has just been filled with the ASCII representation of I. Of course, the ASCII'OF(... FREE) define can also be used in ordinary MOVEs, such as MOVE XXX:="(" CONCAT ASCII'OF (TEMP FREE) CONCAT ")"; In fact, its usage in SAY ... TO'TERMINAL is merely a special case of its usage in MOVEs (remember, SAY ... TO'TERMINAL actually expands out into a MOVE and a PRINT). Similarly, if we take a fancy to DOUBLE integers, we can define DASCII'OF to convert a DOUBLE to a string: INTRINSIC DASCII; DEFINE DASCII'OF = ASCII'BUFFER,(DASCII #; Now, a SAY "D=" CONCAT DASCII'OF (D FREE) TO'TERMINALS; will print "D=" followed by the value of the double integer D. For the curious, FREE stands for "free-formatted", which indicates that it uses as many spaces as needed to fit the number. Next issue, we'll talk about formatting integers/double integers in a fixed-length field (left-justified, right-justified, or zero-filled), outputting in Octal, saying things to files, and lots of other goodies. Tune in next months, same BAT-TIME, same BAT-CHANNEL ... Q: I have an application program to write which will dynamically allocate storage. Since I also want to use SORT/3000, I need to know the minimum amount of stack space required by SORT. Hewlett-Packard seems to believe this information is proprietary. Can you help? A: Sort/3000 has a fixed work area of 2628 words. It also requires twice the area (in words) of one more than the logical record (in words) of the file to be sorted. An additional 256 words will be used if an alternate collating sequence (e.g. sort alpha before numeric) is used. Examples: 1. Record length is 132 bytes or 66 words. (No alternate collation sequence) Minimum stack = ((66+1)*2)+2628 " " = 2762 2. Record length is 123 bytes or 62 words. (No alternate collation sequence) Minimum stack = ((62+1)*2)+2628 " " = 2754 3. Record length is 121 bytes or 61 words. (alternate collation sequence) Minimum stack = ((61+1)*2)+2628+256 " " = 3008 Q: Whatever happened to the PROCINFO intrinsic? It appeared in the April, 1981 update to the Intrinsics manual, then was deleted in a subsequent revision. Are there any other ways to get some of the information it would have provided, such as the name of the currently-executing process? A: When the PROCINFO intrinsic first appeared in the April, 1981 update to the Intrinsics manual, it was not officially supported; as a result, PROCINFO was deleted in the next Intrinsics manual, and did not become an officially supported intrinsic until the CIPER release of MPE. Documentation on this intrinsic can be found in the CIPER communicator, issue number 29. The following are examples of how to use PROCINFO to get the file name of the currently-executing process: SPL: $CONTROL USLINIT,LIST,SOURCE BEGIN BYTE ARRAY PROGRAM'NAME(0:27); ARRAY PROG'NAME(*)=PROGRAM'NAME; INTEGER ERROR1, ERROR2, GET'PROGRAM'NAME:=10; INTRINSIC PROCINFO, PRINT, QUITPROG; <<GET THE PROGRAM NAME - 0 is used for current process>> PROCINFO(ERROR1, ERROR2, 0, GET'PROGRAM'NAME, PROGRAM'NAME); IF <> THEN QUITPROG(ERROR1); <<DID NOT WORK, SO QUIT>> <<PRINT PROGRAM NAME>> PRINT(PROG'NAME,-27,%40); END. COBOLII: $CONTROL USLINIT,LIST,SOURCE IDENTIFICATION DIVISION. PROGRAM-ID. COBTEST. ENVIRONMENT DIVISION. DATA DIVISION. WORKING-STORAGE SECTION. 77 ERROR-1 PIC S9(4) COMP. 77 ERROR-2 PIC S9(4) COMP. 77 GET-PROGRAM-NAME PIC S9(4) COMP VALUE +10. 01 PROGRAM-NAME PIC X(28). PROCEDURE DIVISION. GET-PROG-NAME. *Using 0 in the PIN parm will return info about current prog. CALL INTRINSIC 'PROCINFO' USING ERROR-1, ERROR-2, \0\, GET-PROGRAM-NAME, PROGRAM-NAME. IF ERROR-1 IS NOT EQUAL TO +0 THEN DISPLAY 'ERROR-1 = ', ERROR-1, ' ERROR-2 = ', ERROR-2 STOP RUN. DISPLAY 'PROGRAM NAME = ',PROGRAM-NAME. STOP RUN. Q: We sometimes lose PMAPs of production programs, making analysis of stack display difficult. Is there any way of saving this information on the computer so that we can recall it later, rather than having to keep a file drawer full of PMAPs. A: As of the Q-MIT, SEGMENTER has been enhanced to include most of the PMAP information as part of a program or SL file; this information is called the FPMAP. External references to procedures are not saved in the FPMAP information. This feature can be invoked depending on the conditions of the System-wide flag and the Job/Session-wide flag, along with the new FPMAP and NOFPMAP parameters of the :PREPARE, -PREPARE and -ADDSL commands. Take note that the Command parameters override the Job/Session FPMAP flag and the Job/Session System flag, unless the System flag has been set unconditionally. The System flag can be set to conditional or unconditional inclusion of the FPMAP. If it is set unconditional, all programs or SL segments will have the FPMAP included during the PREP or ADDSL commands, regardless of any other flag or command. If it is set conditional then it can be overridden by lower levels of the FPMAP option control. Thus the System flag and the Job/Session flag determine the default value for the FPMAP option at the command level. For example if the System flag was set conditional (conditional/unconditional applies only to the System flag and only to the "ON" setting) and the Job/Session flag was set "ON" and the NOFPMAP parameter was applied to the PREP command, the result is than no FPMAP would be included in the program module. However, if no FPMAP parameter is specified, then an FPMAP be included in the program module. To display the conditions of the System and Job/Session FPMAP flag, the SHOW command of SEGMENTER is used. To list the FPMAP information of a prepared program or SL segment, the LISTPMAP command of SEGMENTER is used. For more information on these commands and options please consult the "SEGMENTER Enhancements" article in the Q communicator. Q: Why are there only a few questions in this column each month? A: There would be more questions if more people would write them. Q: How do you write a control-Y trap procedure in PASCAL/3000? The System Intrinsics manual says that if control-Y is pressed while the system is executing on your behalf, parameters may be left on the stack. These parameters, it says, must be deleted by the control-Y trap procedure using the "EXIT N" instruction. But, how can you execute the "EXIT N" instruction from a language other than SPL? A: This is a very good question; in fact, until I read it, I myself didn't realize the importance of doing as the manual says and executing the appropriate EXIT instruction. When your process is executing, the computer isn't necessarily executing code in your own program. Often, you'd call various procedures in the system SL -- like FOPEN, FREAD, or compiler library procedures -- and the computer will execute their code on your behalf. In other words, when you do a PASCAL READLN, PASCAL will call a compiler library procedure O'READLN, which in turn will call FREAD. Your process will still be executing, but it will be executing code that resides in the system SL, not your own program. Say that you have enabled control-Y trapping and control-Y is hit. There are two possibilities: either it was hit when your own program code was executing, or when system SL code was executing on your behalf. If your own code was executing, the situation is simple. The computer senses the control-Y and then does the following: * It builds a so-called "stack marker" on the stack to save the place where your program was when control-Y was hit. The stack now looks like: -------------------- | stack marker | <--- System's Q register points here | (4 words) | -------------------- | whatever you had | | on the stack | | (intermediate | | results, local | | variables, etc.) | | are still here | | | * Then it transfers control to your trap procedure, which may � do whatever you like. * Finally, when the procedure is done, a simple "EXIT" instruction (compiled by a PASCAL procedure's END statement) will remove the stack marker and automatically return to where the program was when control-Y was hit. For this, a simple procedure like PROCEDURE TRAP; BEGIN; WRITELN ('CONTROL Y!'); RESETCONTROL; END; would do -- the "END" will automatically return to where you left off, and all will be well. On the other hand, say that the computer was executing system SL code on your behalf. For instance, the control-Y was recognized in the middle of a terminal read done using the READX intrinsic. The computer can't very well immediately transfer control to you -- the READX intrinsic is still executing. It might be holding some System Internal Resources (SIRs), or be in some critical state; in any case, it's nothing that a mere mortal like you should be able to interrupt. Rather, the system waits for the READX to finish, and the moment it finishes and returns control to your program, your trap procedure is called. However, there's one very important catch to this. When you call READX (or the compiler compiles code to call READX on your behalf), all of READX's parameters -- the file number, the buffer address, and the read length -- are put onto the stack. When a control-Y is sensed during an READX, all of READX's code is allowed to finish executing EXCEPT FOR THE PART THAT REMOVES THE PARAMETERS FROM THE STACK! What does this mean? This means that when your trap procedure is called, the stack looks like: -------------------- | stack marker | <--- Q register | (4 words) | -------------------- | READX length | <--- Q-4 | READX buffer addr| <--- Q-5 � -------------------- | whatever you had | | on the stack | | | These extra three words have been put on the stack, and when you return to your main program, they will stay on the stack! Your main program, of course, does not expect them -- it expects, say, an intermediate value in an expression on the top of the stack, but what it gets is the length from an READX call! Let me demonstrate this with an example (run this on your computer to see for yourself): $LIST OFF$ $STANDARD_LEVEL 'HP3000'$ $USLINIT$ PROGRAM TEST (OUTPUT); TYPE SMALLINT = -32768..32767; { needed for READX } VAR DUMMY: INTEGER; INBUFF: ARRAY [1..128] OF INTEGER; PROCEDURE XCONTRAP; INTRINSIC; PROCEDURE RESETCONTROL; INTRINSIC; FUNCTION READX: SMALLINT; INTRINSIC; PROCEDURE TRAP; BEGIN WRITELN ('CONTROL-Y!'); RESETCONTROL; END; BEGIN XCONTRAP (WADDRESS(TRAP), DUMMY); WRITELN (READX (INBUFF, -255)); END. When you run this and don't hit control-Y, the WRITELN will output the number of characters read, which is what READX is supposed to return. However, if you do hit control-Y, the WRITELN will output -255, the parameter you passed to READX! Think about what the stack looks like when READX is called: � -------------------- | -255 | <--- S-0 | INBUFF address | <--- S-1 | room for result | <--- S-2 -------------------- | whatever you had | | on the stack | | | In the normal course of affairs, READX will drop its two parameters (the -255 and the INBUFF address) from the stack, and will leave its result -- the number of characters read -- on the top of the stack. Then, WRITELN will take the value from the top of the stack and print it. However, when control-Y is hit, your procedure will be called without popping the two READX parameters from the stack. Then, when it returns, the parameters will still be unpopped. WRITELN prints the value from the top of the stack and it's -255! Fortunately, the sages of Cupertino gave us a solution to this problem. When your trap procedure is called, it leaves at location Q+1 (the first cell allocated for your procedure's local variables) the number of system intrinsic parameters that have been left on the stack. Then, you can assemble a special "EXIT" instruction that pops those parameters from the stack. The only problem is that there's no way you can assemble and execute a machine language instruction from PASCAL! (And you thought I'd never get around to your question! Fooled you, eh?) Now, there's no doubt that doing as the manual says and executing the special instruction is really necessary. If you don't, the dreadful things that I described above will happen to you. Since SPL is the only language in which this special instruction can be built, you have to write a special SPL procedure. There are two possible approaches to this. The first (suggested by Steve Saunders of HP) is: write an SPL procedure that does nothing except calling the PASCAL error handling procedure and then assembling and executing the EXIT instruction. This way, you'll still be able to write most of your trap-handling procedure in PASCAL. For instance, your SPL procedure might look like: $CONTROL NOSOURCE, SEGMENT=SPL'CONTROLY, SUBPROGRAM, USLINIT � BEGIN PROCEDURE PASCALCONTROLY; OPTION EXTERNAL; << should be in main program >> PROCEDURE SPLCONTROLY; BEGIN INTEGER SDEC=Q+1; PASCALCONTROLY; << the PASCAL proc should do the RESETCONTROL >> TOS:=%31400+SDEC; ASSEMBLE (XEQ 0); END; END. You'd declare SPLCONTROLY as EXTERNAL in your PASCAL program, call XCONTRAP passing to it WADDRESS(SPLCONTROLY), and then have your PASCALCONTROLY procedure invoked by SPLCONTROLY. The SPL procedure takes care of the EXIT and the PASCAL procedure does everything else. The problem with this method is that it requires that the SPL procedure must be included into the same USL as the PASCAL program (otherwise the SPL procedure wouldn't be able to call the PASCAL one). Furthermore, if you want to use the same SPL procedure for all your programs that need control-Y, you'd have to make sure that your PASCAL control-Y handling procedures are all called PASCALCONTROLY (or whatever you decide to call them, so long as it's the same for all programs using the particular SPL procedure). On the other hand, this is an A-1 100% guaranteed super-safe method. Unlike the one I'll get into presently, it will work (hopefully) regardless of the way PASCAL allocates its local variables or uses Q+1. The alternate method is to have a special SPL procedure (that can be put into an RL or SL) that is called from the PASCAL control-Y procedure just as the PASCAL procedure is about to exit. The PASCAL procedure itself is set up as the trap procedure by an XCONTRAP call, and the SPL procedure is only called to do the appropriate stack clean-up. The SPL procedure is: $CONTROL NOSOURCE, USLINIT, SUBPROGRAM, SEGMENT=EXIT'CONY'TRAP BEGIN PROCEDURE EXITCONYTRAP; BEGIN INTEGER DELTAQ=Q-0; INTEGER SDEC=Q+1; << First, we have to move the Q register to where it was in the >> << procedure that called us, which should be the control-Y trap >> << procedure. Note that EXITCONYTRAP must be called DIRECTLY >> << from the control-Y trap procedure. >> << The relative address of the Q register of the calling >> << procedure is kept in Q-0. >> PUSH (Q); TOS:=TOS-DELTAQ; SET (Q); << Now, it's as if we were in the trap procedure itself. >> << Q+1 now contains the stack decrement. >> << Build an EXIT instruction on the top of the stack. >> << An "EXIT N" instruction, which means "exit, popping N words >> << from the stack", is represented by %31400 + N. >> TOS:=%31400+SDEC; ASSEMBLE (XEQ 0); << Execute the instruction on top of stack >> END; END. How can you use this monstrosity? Well, you can put in an SL, RL, or even directly into your PASCAL program's USL. Then, your program can declare it as: PROCEDURE EXITCONYTRAP; EXTERNAL; and call it from your trap procedure right before the "END" statement. EXITCONYTRAP will get rid of the junk on the stack and will exit your PASCAL trap procedure. As was mentioned above, EXITCONYTRAP must be called directly from your trap procedure. Thus, the program we showed above should be re-written as: $LIST OFF$ $STANDARD_LEVEL 'HP3000'$ $USLINIT$ PROGRAM TEST (OUTPUT); TYPE SMALLINT = -32768..32767; { needed for READX } VAR DUMMY: INTEGER; INBUFF: ARRAY [1..128] OF INTEGER; PROCEDURE XCONTRAP; INTRINSIC; PROCEDURE RESETCONTROL; INTRINSIC; FUNCTION READX: SMALLINT; INTRINSIC; � PROCEDURE EXITCONYTRAP; EXTERNAL; PROCEDURE TRAP; VAR SDEC: -32768..32767; BEGIN WRITELN ('CONTROL-Y!'); RESETCONTROL; EXITCONYTRAP; END; BEGIN XCONTRAP (WADDRESS(TRAP), DUMMY); WRITELN (READX (INBUFF, -255)); END. This program WILL work. Even when control-Y is hit, the READX parameters will be popped from the stack, and the WRITELN will output the right data: One bit of mystery here -- what is the "VAR SDEC: -32768..32767" for? Well, it is unneeded in this particular program; however, I strongly suggest that it be included as THE FIRST DECLARATION in all control-Y trap procedures. PASCAL doesn't know that a procedure is a control-Y trap procedure, and if you declare local variables in the procedure, the first of the variables will be allocated at Q+1, which is where the stack decrement was put by MPE. Declaring "VAR SDEC: -32768..32767" allocates the variable SDEC at Q+1, and all subsequent variables will be allocated at Q+2, Q+3, etc. The variable SDEC, by the way, can be called anything -- what is important is that it be the first declaration in the procedure and that you never change it. That way, the contents of Q+1 will remain unchanged until the EXITCONYTRAP procedure is called. This points out the major flaw in this approach: it relies on PASCAL allocating local variables in a particular way -- we have to be certain that we can "reserve" Q+1 for ourselves. If PASCAL decides to allocate local variables in a different, unpredictable way, we'd have to restrict ourselves to having no local variables at all in the procedure, for fear that one of them will step on Q+1. Worse, if PASCAL decides to use Q+1 for its own internal purposes, then Q+1 will be forever lost to us, and the method won't work. However, at the moment PASCAL does NOT use Q+1, and we CAN reserve it by the simple declaration I showed above. In fact, since the current method of variable allocation is fairly standard, being the same in SPL, FORTRAN, and PASCAL, I rather doubt that it will ever be changed substantially. Therefore, I think that the advantages of being able to put this �SPL procedure in an RL or system SL and having your PASCAL control-Y trap procedure be called anything you please more than outweigh the disadvantages of possible future incompatibility. In either case, I think HP should document at least one of these ways of handling control-Y traps from PASCAL. Even better would be a special mechanism by which the PASCAL compiler would do all the dirty work for you -- reserving Q+1 and building and executing the EXIT instruction, possibly triggered by some "CONTROLY" keyword (e.g. "PROCEDURE FOO; CONTROLY;"). If you (the reader) are interested in this, please send in an enhancement request to HP. Q: We have an application program that is composed of one main body and a number of subroutines (one for each main menu option). Unfortunately, the HP's architecture restricts us to having at most 64 segments (of 16K each) in the program, so we can't just compile all the subroutines into one USL file. One possible solution for us was putting some of the subroutines into an SL file. However, each subroutine that is referenced by the program is loaded together with the program itself when the program is loaded; thus, every such segment that we put into an SL uses up a CST entry, of which there are precious few. The answer that we came up with was to put the subroutines into the SL but load them dynamically; in other words, we'd do something like: MOVE "AP020" TO PROC-NAME. ... CALL PROC-NAME USING PROC-PARMS. This way, the AP020 subroutine is loaded only when it is needed; if nobody needs it, it'll never be loaded and won't use the CST entry. Are there any problems with this? A: The problem you mentioned -- the limit of 63 segments/program -- is indeed one that is bothering many people who run large application systems. In fact, MPE V solves this problem by permitting up to 255 segments/ program; by the time you read this, you may already have MPE V and your troubles would then be over. Until you get MPE V, the solution you proposed -- specifying the subroutine name as a COBOL variable -- will work; however, you should watch out for several pretty nasty pitfalls that it can cause. The way COBOL handles the "CALL <variable-name>" construct is by calling the so-called "LOADPROC" intrinsic to dynamically load the subroutine being called. LOADPROC, unfortunately, is very slow it can take about a second or at least an appreciable fraction of a second. Furthermore, loading on the 3000 is serialized -- only one load can be taking place on a computer at any given time. Thus, if three LOADPROCs (or program loads) are being done simultaneously, they'll have to go one after the other, and the last one can take a long time indeed. Also, any process can in its life call LOADPROC no more than about 250 times; any LOADPROCs after that will fail. This can be a problem if you use a lot of these variable name calls. One thing you can do to mitigate both these conditions is by calling the special COBOL "???" subroutine, which loads a given procedure for you and then returns its plabel (a simple integer). Then you can use the plabel to call the subroutine. In other words: 77 PROC-PLABEL PICTURE IS S9(4) COMP-3. ... * Use this CALL before calling your subroutine the first time. CALL "???" USING PROC-NAME, PROC-PLABEL. ... * Use this CALL for all your subroutine calls. CALL PROC-PLABEL USING PROC-PARMS. This way, the subroutine will have to be LOADPROCd only once in the life of your process -- this will still be inefficient, but it's better than loading it every time you want to call it. If you don't know at compile-time in which place a given procedure will be called first, you can always initialize PROC-PLABEL to 0 and check PROC-PLABEL before every call; if it's 0, you'll call "???" and then do a "CALL PROC-PLABEL" -- otherwise, you'll just do a "CALL PROC-PLABEL". Of course, every procedure you load will have to have its own plabel. As you can see, all this is a mess -- even at best, LOADPROC is quite inefficient, and the "CALL <variable-name>" construct which uses it is quite inefficient. I suggest that if you can, wait for MPE V to arrive -- with it, as I said, all your problems (well, not all, but at least this one) will go away. One other thing for people who use LOADPROC explicitly (i.e. actually call it as an intrinsic from SPL, FORTRAN, PASCAL, etc.): once you load a procedure, it never really gets unloaded until the program that loads it terminates. Even if you do an UNLOADPROC, the procedure will still use CST entries, and the same maximum of 250 LOADPROCs per lifetime of the process still remains in force. Q: The new (MPE V) JOBINFO intrinsic allows me to find out what job or session is logged on under a particular user id (e.g. CLERK.PAYROLL). Unfortunately, there is often more than one session on the computer with the same user id and session name -- JOBINFO only gives me information on one of them. How can I find all the sessions that are logged on under a particular user id? A: Indeed, JOBINFO does not nicely handle this condition. It is ironic that HP took great pains to make this intrinsic extendable -- new options can easily be added to it that will permit it to return additional information -- but failed to properly take care of a common condition that can and does occur today. Fortunately, there is a fairly simple (well, at least not very complicated) solution to this problem -- you can redirect the :SHOWJOB listing into a disc file just as you would a :LISTF output! In other words, you can execute the following three commands: :FILE MYFILE,NEW;SAVE;REC=-80,,F,ASCII;NOCCTL :SHOWJOB JOB=CLERK.PAYROLL;*MYFILE :RESET MYFILE and MYFILE now contains the :SHOWJOB output for all the sessions signed on as CLERK.PAYROLL. All your program needs to do is to execute these commands using the COMMAND intrinsic, open MYFILE, and then read it, picking up the job numbers and session numbers as it goes along. Then, you can use the job and session numbers to call JOBINFO to get more detailed information; or, perhaps, the information in the :SHOWJOB line would be enough for you and you wouldn't even need to call JOBINFO. Interestingly enough, although this feature has been present at least since the Q-MIT -- perhaps since the CIPER MIT or even since the very first MPE IV release -- it is at best sparsely documented; it is certainly not documented in MPE V :HELP, and I believe it isn't mentioned in the manuals, either. Finally, since you've asked about JOBINFO anyway, I might caution you about a few problems I've found using it. I found these on MPE version G.B0.00, which I believe is the commonly released version of MPE V. * I couldn't get JOBINFO to return to me information on a particular user.account SESSION. I could get information on a particular JOB quite nicely (by specifying the job's logon id as the first triplet), and I could get information on a job or session given the job/session number, but I couldn't get information on a session given the logon id. It insisted on returning information on my own logon session. * Whenever I retrieved the logon id (item 1) of a particular job or session THAT DIDN'T HAVE A JOB/SESSION NAME, the logon id was not blank-padded; instead, the several characters after the account name were filled with garbage! Even if I filled the output buffer with spaces before calling JOBINFO, the garbage would still appear. I suggest that instead of trying to retrieve the full logon id, you get the job/session name, user name, and account name (items 2, 3, and 5) and assemble them yourself. * Finally, the input and output device numbers/class names (items 9 and 10) are not blank-padded like they should be. Fortunately, they're not garbage-padded either -- if you fill the output buffer with spaces before calling JOBINFO, you'll get a blank-padded result. I hope that these bugs will be fixed by HP in the near future; in the meantime, the workarounds I indicated, especially the :SHOWJOB into a disc file, should tide you through. Q: When we run DBUNLOAD or DBLOAD from a session, we get a message every time a dataset has been finished, and can thus easily monitor the progress of the operation. However, if we run those programs from a job stream, the "dataset finished" messages go to the spool file, which we can't read until the job stream is done. If this were DBSCHEMA, for instance, that we were running, we could easily send the program's output to a file by using the "formal file designator" DBSLIST on a file equation. For instance, :FILE DBSLIST=FOO,NEW;SAVE;DISC=4095,32;DEV=DISC;ACC=OUT :RUN DBSCHEMA.PUB.SYS;PARM=2 will send the DBSCHEMA output to FOO. If we could do this to DBLOAD and DBUNLOAD, we could send the output to some disc file which could then be looked at from a session. Unfortunately, redirecting DBSLIST does not work with DBLOAD or DBUNLOAD. What other way is there to make the DBLOAD/DBUNLOAD output go to a disc file? A: To answer your question, I'd like to first bring up some interesting facts about the history of the 3000. In the dark, distant days of the earlier '70s, when the HP3000 was but a wee tot in its first few years of life, not much thought was given to redirection of program output. If a running program, for instance, called the PRINT intrinsic, or FWRITE against $STDLIST, the data that it writes would ALWAYS go to your terminal (or the spool file, for jobs). No ifs, ands, or buts -- there was no way of redirecting this output. Of course, for many programs, such as the compilers and DBSCHEMA, there had to be some way of sending output somewhere else (although this was usually the line printer or some such device, not a disc file). So, HP decided to use :FILE equations to accomplish this. To open a file, a program must -- either directly or indirectly -- call the FOPEN intrinsic, passing to it various information such as the file's name, device, foptions, etc. Then, if there is any :FILE equation existing for the filename specified in the FOPEN call, the parameters specified on the :FILE equation supersede those passed to FOPEN. Thus, DBSCHEMA opens its list file with the name "DBSLIST" and the foptions indicating that the file should by default be "$STDLIST". If there's a :FILE equation for DBSLIST that redirects it to, say, the line printer or a disc file, the :FILE equation will take precedence and the printer or file will be opened. This, incidentally, is why the first parameter passed to FOPEN (in the case of the DBSCHEMA list file, this parameter contains "DBSLIST") is called the "formal file designator" rather than just the "filename". The formal file designator is not necessarily the real filename; rather, it is the default filename which can be redirected elsewhere using a :FILE equation. The formal file designator/:FILE equation approach was thus an admirable solution to the problem -- by default the output would go to the terminal, but it could be redirected by the user to go anywhere else. Unfortunately, this solution had a very substantial problem of its own -- it only worked when the program actually called FOPEN to open its list file rather than called PRINT directly. Consider DBLOAD and DBUNLOAD. Their authors probably did not see why people would ever want to redirect their output. Of course, you've just mentioned a very good reason for this kind of redirection, but the authors didn't anticipate it. So, not expecting the redirection problem to ever show up, they did all their output not by opening an output file and then writing to it, but rather by directly calling the PRINT intrinsic. Since the PRINT intrinsic outputs things directly to the terminal (or spool file), a :FILE equation wouldn't work to redirect the display. For a long time -- up until around 1981 -- there were many programs that could never have their output redirected precisely because they were not written with redirection in mind and thus directly called the PRINT intrinsic. Around 1981, with the ATHENA MIT, HP finally decided to adopt a more more thorough solution. You're probably familiar with it already -- the :RUN command now permits you to redirect the output (and input) of any program by specifying a ;STDLIST= or a ;STDIN= parameter. Thus, you no longer need to have the program provide a formal file designator (like DBSTEXT) for which you can issue a :FILE equation. You can force even a non-cooperating program to send output to a file (or accept input from a file). Thus, for your DBLOAD or DBUNLOAD call, you only need to say: :BUILD MYFILE;REC=-80,,,ASCII :FILE MYFILE,OLD;SAVE;ACC=OUT;SHR;GMULTI :RUN DBLOAD.PUB.SYS;STDLIST=*MYFILE and the DBLOAD output will be redirected to the given file. In fact, you can do this even to a program that already permits redirection via a formal file designator; for instance, DBSCHEMA's output can be redirected by saying :BUILD MYFILE;REC=-80,,,ASCII :FILE MYFILE,OLD;SAVE;ACC=OUT;SHR;GMULTI :RUN DBSCHEMA.PUB.SYS;STDLIST=*MYFILE instead of :BUILD MYFILE;REC=-80,,,ASCII :FILE DBSLIST=MYFILE,OLD;SAVE;SHR;GMULTI :RUN DBSCHEMA.PUB.SYS;PARM=2 Thus, for all practical purposes, formal file designators are now obsolete, since anything you could do with them you can do as well or better with ;STDLIST= and ;STDIN=. Finally, note that I didn't just say :FILE MYFILE,NEW;SAVE :RUN DBLOAD.PUB.SYS;STDLIST=*MYFILE but rather executed a :BUILD command and added a number of extra parameters to the :FILE equation. This is because the simple :FILE/:RUN approach will not actually save the file as a permanent file until the program is done. In the :BUILD/:FILE/:RUN approach, the :BUILD builds the file before the program even starts up, and the "SHR" and "GMULTI" parameters on the :FILE make sure that you'll be able to read the file from another session. Q: I would like to :ALLOW some spooler commands to a particular user. Unfortunately, the :ALLOW command cannot permanently allow things to a particular user, only to a particular session or to everybody. How can I do this without having to have the console operator do an :ALLOW user.account;COMMANDS= every time the given user logs on? A: It is these little quirks in the way HP does things that make life interesting for us HP users. Imagine, if HP did everything right the first time, where would all us independent software vendors (and contributed-library program authors) be? ADAGER, QEDIT, MPEX -- all these independent vendor products were made to remedy deficiencies in the 3000 that by rights shouldn't have been there in the first place. One such problem is the fact that, unlike capabilities -- which stay with a user until they are explicitly removed or altered, no matter how many times he may log on or off -- allows (except system-global allows) stay with a user only for a single session. Fortunately, there is a program in the contributed library called ALLOWME that, when run as a logon UDC, will read a file that contains user names and :ALLOWed commands, and will :ALLOW to the current session the commands to which the user logged on as this session is entitled. Thus, you could build the following file (this is just a rough sketch -- for the exact details, see the ALLOWME documentation in the contributed library): MANAGER.DEV DELETESPOOLFILE, ABORTIO TEST.DEV DELETESPOOLFILE MANAGER.PROD ABORTIO, ABORTJOB, DOWN and add a command to your UDC to automatically run ALLOWME at logon. ALLOWME (which is, of course, privileged) will recognize what commands are allowed to the user, and will grant them to him for the rest of the session. Finally, let me also mention another command to you called ":ASSOCIATE". This command makes a particular user be the "console operator" for a particular LDEV. All messages pertaining to that LDEV will be sent to the user, and all operator commands working with the LDEV will be allowed to that user. This is useful if, for instance, you have a remote printer and you want some user other than the console operator to be able to :REPLY to requests on it, start spooling on it, etc. Finally, another relevant command is ":JOBSECURITY LOW". It essentially permits any user to abort his own jobs, and any account manager to abort any jobs in his account. This is often much better than the default (":JOBSECURITY HIGH") state, in which only the console operator can abort it. If you're :ALLOWing people the :ABORTJOB command, you ought to consider doing a ":JOBSECURITY LOW" instead. The same, incidentally, applies to ":ALTJOB", ":BREAKJOB", and ":RESUMEJOB". Q: We do our system cleanup -- things like a VINIT >CONDense and such -- at night without operator assistance. However, some people sometimes leave their sessions logged on overnight, and sometimes jobs are started up in the evening that are still running when we want to do the cleanup. We'd like to be able to automatically abort all the jobs and sessions in the system (except, of course, the job doing the aborting). How can we do this? A: (The solution to this problem was provided by Vladimir Volokh, President, VESOFT, Inc.) What we would really like to have -- what would make the problem trivial to solve -- is a command of the form :ABORTJOB @.@ Try finding that one in your reference manual. The sad fact, of course, is that no such a command exists. However, we do have two commands that each do about half of the task: * :SHOWJOB, which finds all the jobs in the system. * :ABORTJOB, which aborts a single job. The trick is to take the output of the :SHOWJOB command and feed it to the :ABORTJOB command, so all the jobs shown are aborted. Without much further ado, here's the job stream we want: !JOB ABORTALL,MANAGER.SYS;OUTCLASS=,1 !COMMENT by Vladimir Volokh of VESOFT, Inc. !FILE JOBLIST;REC=-80;NOCCTL;TEMP !SHOWJOB;*JOBLIST !EDITOR TEXT JOBLIST DELETE 1/3 DELETE LAST-6/LAST FIND FIRST WHILE FLAG DELETE "MANAGER.SYS" CHANGE 8/80,"",ALL CHANGE 1,":ABORTJOB ",ALL KEEP $NEWPASS,UNN USE $OLDPASS EXIT !EOJ A brief explanation: * First we do a :SHOWJOB of all the jobs in the system to a temporary disc file (note that this redirection of :SHOWJOB output to a disc file might not be documented -- it certainly isn't documented in :HELP, nor is it mentioned in some of the older manuals). * Then we enter editor and massage the file so that it contains only the lines pertaining to the actual jobs (all the lines except the first 3 and the last 7). Then, we exclude all the jobs signed on as MANAGER.SYS, since we don't want to abort our system cleanup job streams (like ABORTALL itself or the job that streamed it), and they are presumably signed on as MANAGER.SYS. Of course, you can easily change this to any other user name. * Now, we strip all the data from the lines except for the job number, and then we insert an :ABORTJOB in front of the number. The file now looks like: :ABORTJOB #S127 :ABORTJOB #S129 :ABORTJOB #J31 :ABORTJOB #S105 :ABORTJOB #J33 * Finally, we keep this as a disc file (in our case, $NEWPASS), and then we /USE this file, causing each of its commands to be executed as an EDITOR command. Since ":ABORTJOB" is a valid EDITOR command -- it just makes the EDITOR call the COMMAND intrinsic -- all these commands get executed! * Note that in order for this job stream to work properly, the :ABORTJOB command must be :ALLOWed to MANAGER.SYS. This can be done via the :ALLOW command or by use of the ALLOWME program in the contributed library (see the answer to the previous question for more information). And there's your answer. Fairly simple, no need to go into privileged mode, no need to even write a special program (other than the job stream itself). A perfect example of "MPE PROGRAMMING" -- performing complicated system programming tasks without having to write a special program in SPL or some such language. Incidentally, for more information on MPE Programming, you can read our paper called -- you guessed it! -- "MPE Programming", which has been printed in the HPIUG Journal, APR/JUN 83 (Vol.6,No.2), or can be gotten from this very EBOOK.VECSL group of files. Q: We are research facility that needs to bill connect and CPU time usage. Unfortunately, we cannot use HP's standard group/account billing system because billing must be done to projects, not individual groups and accounts. Members of different projects can log on to the same group/account, and members of the same project can log on to different groups and accounts. One solution that we contemplated is to have a logon UDC that asks the user for his project number and logs it and his logon time to a disc file. Then, a logoff UDC records the total connect and CPU time used in the same file. Two problems, however, confront and confound us: for one, there's no such thing as a "logoff UDC", in fact no way at all to require something to be done when the user logs off; and, there seems to be no way of getting a job's total CPU usage. Having come to the end of our rope upon the deserted island of our despair, we enclose this cry of help into a bottle and entrust it to the whims of the United States Postal Service... Signed, Marooned without a Billing System Dear Marooned: Both of the problems that you mentioned are pretty nasty ones, the "logoff UDC" one more so than collecting the total CPU time used by a session. Fortunately, there is a solution that lets HP do the dirty work. It involves using the HP logging system. Enable JOB INITIATION and JOB TERMINATION logging, and require all your users to sign on with their project number as part of their session name. Then, by combining the data from the initiation and termination records, you can figure out which project was using what. The only thing you really need to concern yourself with is that all people provide the right project numbers -- you must protect yourself not only from malice, but also from honest error caused by people mistyping a session name or omitting it entirely. You can trace back and appropriately chastise the culprits by looking at the user, account, and terminal number specified in the job initiation log record. Even better, you can write a small program (to be put into your logon UDC) that will get the session name and make sure that it is legitimate -- perhaps even ask for some kind of "project password". To get the session name, you might use the new JOBINFO intrinsic (if you have MPE V/E); if you don't, call me at VESOFT ((213) 937-6620, or write to the address given above) and I'll send you a copy of a procedure that does it. If you by any chance already use session names for something else, you can always have a logon UDC that inputs the project name and writes it into a file; then, you can combine the data from this file and the system log files. Q: Sometimes we have a program file and aren't sure which version of source code it came from. Usually, we have a strong suspicion which one it is, so we tried to simply recompile the suspected source file and then use :FCOPY ;COMPARE to compare the compiled source with the program file. Unfortunately, FCOPY gave us hundreds of discrepancies; in fact, we found that the program file changed every time it was run! What can we do? A: As you pointed out, every time a program is run, various parts of it -- record 0, the Segment Transfer Tables (STTs) in each segment, etc. -- are changed. FCOPY is therefore a dead end, since it's virtually impossible to tell a legitimate discrepancy from a difference caused by the loader. However, there are some things you can do. First of all, you can of course :LISTF both program files -- if they are of different sizes, they're clearly substantively different. Of course, if they're of the same size, you can't be certain that they are actually the same. Beyond that, your best solution is to have some kind of version numbering system. If you can be sure to increment a version number variable in your source every time you make a change, and then print the version number whenever you run the program, you can very easily tell whether two program versions are the same just by looking at their version numbers. The only problem with this is that it's rather easy to forget to change the version number. If you think this will happen often, you could try some more automatic but less reliable approaches. For instance, you could simply compare the creation dates of the program file and the source file; since whenever you /KEEP a file in EDITOR, it's re-created, thus updating its creation date, it's a pretty good bet that if the two creation dates are equal, you're dealing with the same version. Of course, this approach is hardly foolproof -- either file might have been copied, thus changing its creation date, or two changes might have been made on the same day. Unfortunately, this is as precise as you're going to get without some kind of version numbering system in the source code. Q: One of my programs keeps aborting with a FSERR 74 ("No room left in stack segment for another file entry") on an FOPEN. True, I do have a procedure that allocates a very large array on the stack -- I run my program with ;MAXDATA=30000 -- but I don't do the FOPEN at the time I'm in the procedure, so at FOPEN time my stack can't be larger than about 15K. What am I doing wrong? A: To answer this question, I have to digress for a moment and discuss the structure of your stack. All of the variables and arrays you declare in your program -- local or global -- are put onto your stack, which is stored by MPE in a single data segment. Now, you don't view your stack as a data segment, since you don't need to call DMOVIN or DMOVOUT to access it; however, deep down inside the system does, and places on the stack a fundamental restriction common to all data segments -- no data segment may ever be more than 32K words long. Now, your stack is actually partitioned into several pieces, each piece pointed to by a machine register: High memory Z > --------------------------------------------- ^ | unused space | ^ S > --------------------------------------------- ^ | operands of machine instructions | ^ | and local variables of the currently | ^ | executing procedure | ^ Q > --------------------------------------------- ^ | local variables of other procedures | ^ | and stack markers indicating calls from | ^ | one procedure to another | ^ Qi > --------------------------------------------- positive | global variables | addresses DB > --------------------------------------------- | "DB negative area," accessible only by | negative | SPL procedures (like V/3000) -- usable | addresses | for global storage | v DL > --------------------------------------------- v | The Nether Regions, where mortals may | v | not stray and non-privileged accessors | v | are punished with bounds violations | v --------------------------------------------- v Low memory (The above picture reproduced with permission from "The Secrets of Systems Tables... Revealed!" by VESOFT. Said permission was not especially difficult to obtain.) Now your stack contains all of this stuff, from the bottom of the Nether Regions to your Z register. The Nether Regions (the DL negative area, also knows as the PCBX) is where the file system allocates miscellaneous information about your file; when you open enough files, the initial space allocated below DL is exhausted, and the system has to allocate some more. Now, if this would make the total size of the stack data segment larger than 32K, you get an FSERR 74. The problem is that the data segment size is not measured from the bottom of the PCBX to the S register, but rather to the Z register. Your S register points to the current top of your stack, so if you're using 15K, your S register value is 15K; however, your Z register points to the highest place that the S register has ever been! Thus, say your stack size starts out at 10K. Your S points to 10K, and your Z points to about 12K (since the system allocates about 2K of overhead between S and Z). Now, you call your procedure which allocates an 18K local variable; now your S is at 28K and Z is at 30K. When you exit the procedure, the S is reset to 10K, but the Z stays at 30K! This leaves you only 2K for the PCBX, even though you have 20K of unused space between S and Z. The solution? Well, I think that your best solution is simply to call the ZSIZE intrinsic, passing to it the parameter 0, before calling FOPEN. This call frees any space you may have between S and Z, thus leaving the maximum possible space for the file system to work with. But, you might say, if the file system allocates an extra 1K in the DL negative area, this will decrease the maximum Z value to 29K, and so when I next try to allocate that 18K array, I'll get a stack overflow! Right? Wrong. It turns out that the highest value I could get Z to take was 30848 (use this number for comparison, your mileage may vary with your MPE version). Thus, if you allocate enough space for S to be pushed up to 28K, Z will be set to 30848; however, if you push S all the way up to 30K, Z will be left at the same value of 30848 without any adverse effects. Thus, you can contract the S-Z area by calling "ZSIZE (0)", call FOPEN, have it allocate a couple of hundred extra words in the DL negative area, and still be able to push S and Z up by 18K with no problems! So, doing a "ZSIZE (0)" before an FOPEN will probably solve your problem (I do this in my MPEX program all the time). However, if it doesn't -- if you are really using every word of stack space and can't fit both your own data and the file system's file information into one 32K stack -- there is an alternative; you may run your program with ;NOCB, which causes most of the file system information to be allocated in a separate data segment. The reason why I didn't suggest this first was that this slightly slows down file system accesses; furthermore, if your program doesn't run without ;NOCB, you can bet that half the time the user will forget to specify ;NOCB and will get an error. I think that the ZSIZE call, if it works, is the cleaner solution. Q: What effect does virtual memory size have on the performance of my system? Can I configure it too large? Too small? A: None. No. Yes. What? You want to know more? I gave you the answers, didn't I? Oh, all right. If you have too little main memory, this will undoubtedly affect your system performance because there'll be a lot of swapping. However, let's pretend that there was no virtual memory -- that if you ran out of main memory, you'd simply get an error message (instead of getting an unused segment swapped out to disk). In that case, the size of main memory would actually have no effect on the performance of your system, if all you mean by performance is the speed at which the system runs. If you configure your main memory too small, you'll simply get an error message when you try to overflow it; however, whatever can run in the reduced memory will run as fast as it would in a configuration with a lot of memory. That's exactly what happens with virtual memory. If virtual memory gets full, you'll just get an error message; then, you can reconfigure your system (unfortunately, it involves a reload) to have more virtual memory, and that's that. If you want to check how much virtual memory you're actually using, you can run the contributed utility TUNER. So, as long as you don't run out of virtual memory, changing its size will not affect your system performance one iota -- the only problem with making it too large is that you waste some disc space (which is fairly cheap anyway). The System Operation and Resource Management Manual suggests that you determine the amount of virtual memory you will need as follows: estimate the average number of concurrent users, the average stack size, the average number of buffered files open for every user, and the number of users who will be simultaneously running programs; then use these figures in conjunction with the values listed below to calculate the amount of disc space to reserve for virtual memory: * 32 sectors for the Command Interpreter stack * approximately 8 sectors for all unbuffered files, depending on buffer size * 4 sectors for every open buffered file * 16 sectors for the system area in the user's stack, plus 4 sectors for every 512 words in the DL/Z area of the stack * 40 sectors for each program being loaded in the system The amount of virtual memory calculated by the above is adequate for the overwhelming majority of systems. If you actually run out of it, just increase it on your next reload. Q: Rumor has it that there is a hardware clock in the Mighty Mouse (Series 37). How can I get to it? A: Getting the Hardware Time Of Century Clock (as it is officially known) is no big deal. Simply put, there is an instruction called "RTOC" (Read Time Of Century) that, when executed from privileged mode, will push onto the stack a doubleword representing the current hardware time of century. You'd use it like this: DOUBLE TIME'OF'CENTURY; ... GETPRIVMODE; ASSEMBLE (CON %020104; CON %7); GETUSERMODE; TIME'OF'CENTURY:=TOS; Note that since the SPL assembler does not know about this instruction, we can't simply say ASSEMBLE (RTOC); Rather, we must specify our instruction by its actual numeric value, which is a %20104 followed by a %7 (since this is a doubleword instruction, which takes two words to represent). The SPL ASSEMBLE (CON x); construct simply causes the number "x" to be put directly into the compiled code; so, if we know the numeric value of an instruction, we can always execute it this way. So, all things considered, the hardware time of century is rather simple to get at -- if what you want is the current date and time represented as the number of seconds since midnight, 1 November 1972. For, indeed, this is yet another chapter in the sad, sad story of computer date and time incompatibility. What you really want is the time and date in some useful format, like month, day, year, hour, minute, and second. At the very least, you'd want it in CLOCK and CALENDAR format. Doing the conversion is far more difficult than getting the value itself. In brief, what we'd want to do is: * Divide the number of seconds (which we call SECS'SIncE'BASE, where "base" refers to 12:00 AM, 01 NOV 1972) by the number of seconds in a day (which is 24*60*60=86400). The quotient is the days since the base (DAYS'SIncE'BASE), and the remainder is the seconds since midnight of the current day (SEC'OF'DAY). * Add 26603 to DAYS'SIncE'BASE to get the so-called "century date", which is the number of days since the mythical 0th of January, 1900. Why 26603? Well, 26603 is a well-known Magic Number, universally revered for its thaumaturgical and necromantic properties. * Now, convert the century date (CENT'DATE=DAYS'SIncE'BASE+26603) to a CALENDAR format date using the handy-dandy procedure given below (and you don't even have to pay me a royalty for using it!) * Finally, convert SEC'OF'DAY to a CLOCK format time by splitting it up into the hour, minute, and second part. To wit: $CONTROL NOSOURCE, USLINIT BEGIN << This program does a Read Time Of Century instruction and then prints out the result in "DATELINE" format. >> INTRINSIC PRINT; INTRINSIC FMTDATE; ARRAY BUFFER(0:12); INTEGER IDATE; DOUBLE DTIME; LOGICAL PROCEDURE D'CAL'TO'CENT (CAL); VALUE CAL; LOGICAL CAL; OPTION CHECK 3; BEGIN << Given the CALENDAR-formatted CAL, returns its corresponding "century date" = the number of days from 00 JAN 00 until CAL. >> DEFINE YEAR = CAL.(0:7) #; DEFINE DAY = CAL.(7:9) #; D'CAL'TO'CENT := YEAR*365+ << # of days in non-leap years >> LOGICAL(INTEGER(YEAR-1)/4)+ << # of leap days >> DAY; END; LOGICAL PROCEDURE D'CENT'TO'CAL (CENT); VALUE CENT; LOGICAL CENT; OPTION CHECK 3; BEGIN << Given the century date CENT, returns the CALENDAR date. >> INTEGER RESULT = D'CENT'TO'CAL; DEFINE YEAR = RESULT.(0:7) #; DEFINE DAY = RESULT.(7:9) #; INTEGER QUADYEAR; << The century may be conveniently broken into groups of 4*365+1 = 1461 days = number of days in 4 contiguous years. Days 1-1461 would be in quadyear 1, 1462-2922 in 2, etc. The only exception to this is that quadyear 1 actually includes days 1-1460 (since 1900 wasn't a leap year). This actually makes things simpler, since then QUADYEAR is simply CENT/1461. Then CENT MOD 1461 is the number of the day in the quadyear; days 0-365 are in year 0, 366-730 in year 1, 731-1095 in year 2, and 1096-1460 in year 3. Note that year 0 has 366 days and all the others have 365. Thus, the year within the quadyear is ((CENT MOD 1461)-1)/365, utilizing the fact that -1 (corresponding to 01 JAN of the 0th year) divided by 365 is 0, and the day within the year is simply CENT-D'CAL'TO'CENT (day 0 of the calculated year). >> QUADYEAR:=CENT/(4*365+1); YEAR:=QUADYEAR*4+INTEGER((CENT MOD (4*365+1))-1)/365; DAY:=0; << now RESULT is day 0 of the right year >> DAY:=CENT-D'CAL'TO'CENT (RESULT); END; PROCEDURE READMMCLOCK (IDATE, DTIME); INTEGER IDATE; DOUBLE DTIME; BEGIN << Reads the hardware clock on a Mighty Mouse and returns the CALENDAR-format today's date in IDATE and the CLOCK-format current time in DTIME. >> INTRINSIC GETPRIVMODE; INTRINSIC GETUSERMODE; DOUBLE SECS'SIncE'BASE; << seconds since 01 NOV 1972 12:00 AM >> INTEGER DAYS'SIncE'BASE; << days since 01 NOV 1972 >> DOUBLE SEC'OF'DAY; << seconds since midnight today >> INTEGER CENT'DATE; << days from 00 JAN 1900 to today >> EQUATE BASE'CENT'DATE=26603; << century date of 01 NOV 1972 >> INTEGER ARRAY ITIME(*)=DTIME; GETPRIVMODE; ASSEMBLE (CON %020104; CON %17); GETUSERMODE; SECS'SIncE'BASE:=TOS; DAYS'SIncE'BASE:=INTEGER(SECS'SIncE'BASE/(24D*60D*60D)); SEC'OF'DAY:=SECS'SIncE'BASE MOD (24D*60D*60D); CENT'DATE:=26603+DAYS'SIncE'BASE; IDATE:=D'CENT'TO'CAL (CENT'DATE); ITIME(0).(0:8):=INTEGER(SEC'OF'DAY/(60D*60D)); ITIME(0).(8:8):=INTEGER((SEC'OF'DAY/60D) MOD 60D); ITIME(1).(0:8):=INTEGER(SEC'OF'DAY MOD 60D); ITIME(1).(8:8):=0; END; READMMCLOCK (IDATE, DTIME); FMTDATE (IDATE, DTIME, BUFFER); PRINT (BUFFER, -27, 0); END. I sincerely hope that Spectrum has a full set of HP-provided intrinsics that convert from various time/date formats to others; there must be hundreds (thousands?) of user-written time and date conversion routines in use on the 3000 right now, with more being written every day. [Thanks to Vladimir Volokh, President, VESOFT Inc. for the answer to the following question:] Q: I am but a humble COBOL programmer only recently introduced to the ecstasies of SPL. I recently wrote an SPL procedure that I want to call from a COBOL program, and it doesn't work quite as I'd like it to. Instead of returning to me the byte array (PIC X(40)) that it's supposed to, it gave me only the last 30 characters of the array, and overwrote some of my COBOL variables that weren't even specified in my procedure call! My SPL procedure looks like: PROCEDURE SUPERWHAMMO (RESULT); BYTE ARRAY RESULT; ... and my COBOL program like: 77 RESULT PIC X(40). ... CALL "SUPERWHAMMO" USING RESULT. Where did I go wrong? A: It seems logical enough that when you have an "X(40)" (which is, after all, an array of 40 bytes) in your COBOL program, the corresponding SPL procedure that sets it should declare it as a "BYTE ARRAY". However, this is not so. COBOL (at least COBOL-68) has no way of knowing what types of parameters your SUPERWHAMMO procedure takes. It knows how many parameters there are (one), since it knows how many you specified in your call. But are they integers? Bytes? By reference? By value? Does the procedure return a result? Is it OPTION VARIABLE? COBOL can know none of these things; however, since it has to pass the parameters somehow, it makes some assumptions about the called procedure. * For one, all of the parameters must be passed by reference -- that means that none of them can be declared as "VALUE xxx". * Furthermore, since passing by reference means passing an address, and addresses might be either word addresses or byte addresses, COBOL arbitrarily assumes that all the parameters are passed as word addresses (i.e. INTEGER, LOGICAL, or DOUBLE, never BYTE). * Finally, it assumes that the procedure returns no result and is not OPTION VARIABLE. If you want to write an SPL procedure that is callable from COBOL-68, you MUST FOLLOW THESE RULES. In other words, your procedure should look like: PROCEDURE SUPERWHAMMO (RESULT'I); INTEGER ARRAY RESULT'I; ... and if you want to treat RESULT'I as a byte array inside the procedure, you ought to say BYTE ARRAY RESULT(*)=RESULT'I; inside the procedure to equivalence the byte array RESULT with the integer array RESULT'I. Incidentally, these restrictions that I outlined are the reason why all the IMAGE intrinsics take INTEGER ARRAYs for parameters instead of BYTE ARRAYs (and are thus such a bother to call from FORTRAN, SPL, or PASCAL) -- the authors of IMAGE knew that most HP users were COBOL users and built the procedures for compatibility with COBOL. Now, the reason for your strange problem -- the procedure returning only the last 30 characters of the array and also overriding other COBOL variables -- becomes apparent. Say that in your COBOL program, RESULT was allocated at word address DB+10. This is 10 words above DB, which is 20 bytes above DB. Therefore, when COBOL passed RESULT's address to SUPERWHAMMO (assuming all along that SUPERWHAMMO was expecting a word address), it passed the value 10. SUPERWHAMMO, however, thinks that this is a byte address, and starts storing data at byte address 10 (which is word address 5). Thus, the area from DB+5 to DB+9 is overwritten with the first 10 bytes of the result. Furthermore, the area from DB+10 to DB+39 (which is where you expect the returned data to be) contains only the last 30 bytes of the returned data, since the returned data actually started at DB+5. By the way, in COBOL-74 (known to friends as "COBOLII"), life is a lot easier. Simply by prefixing your parameter with an "@", as in CALL "SUPERWHAMMO" USING @RESULT. you can tell COBOL-74 that SUPERWHAMMO expects a BYTE ARRAY as a parameter (similar things can also be done to indicate by value parameters and returned results). Thus, CALL "SUPERWHAMMO" USING RESULT. is compatible with PROCEDURE SUPERWHAMMO (RESULT); INTEGER ARRAY RESULT; in either COBOL-68 or COBOL-74, and CALL "SUPERWHAMMO" USING @RESULT. is compatible with PROCEDURE SUPERWHAMMO (RESULT); BYTE ARRAY RESULT; in COBOL-74. Q: I have an SPL program with a control-Y trap routine that stops the program when control-Y is hit. Now, I'd like to be able to do something similar when the program is running in batch -- abort the program from my on-line session. However, a simple :ABORTJOB just won't do, since the program must do some cleanup before it terminates. Is there any way known to man or Guru to pass a 'control-Y' to my program when it is being run in batch mode? A: Of course, the Guru knows everything, and is never at a loss for an answer. What you really want is some way of telling a program (be it running online or in batch) to stop whatever it's doing and perform some trap routine (which might abort the program, print some kind of messages, or whatever). Control-Y is one way of doing this; however, it works only online and only with the program that is running on the same terminal as the one where control-Y is hit. Another method is so-called "soft interrupts". Just like XCONTRAP allows you to tell the system to call a certain procedure whenever control-Y is hit, soft interrupts allow you to cause a procedure to be called whenever a NO-WAIT I/O against a message file completes. Thus, your batch program could do the following: * Build a new, empty, permanent message file. * FOPEN it with NO-WAIT I/O (this doesn't require PM capability). * Call the FINTSTATE intrinsic, passing to it TRUE -- this enables soft interrupts. * Call FCONTROL mode 48, passing to it the plabel of the procedure which you want triggered when a record is written to the file. * Call FREAD to start the nowait I/O. Then, the moment a record is written to the file -- say, by your online session -- the trap procedure is called by the program. This trap procedure might cause the program to stop, or possibly do a :TELLOP to the console indicating what it's currently doing, or any one of a number of other things. Message files being one of the best-documented portions of the file system (as well as best-written, which we owe primarily to a certain HP Lab engineer named Howard Morris), this mechanism is described in more detail by the MPE File System Reference Manual (see section on 'software interrupts'). Q: My job stream looks like: :JOB ... ... :CONTINUE :FCOPY FROM=MYFILE1;TO=MYFILE2;NEW FROM=MYFILE3;TO=MYFILE4;NEW EXIT :IF JCW<>0 THEN : TELLOP SOMETHING IS ROTTEN IN THE STATE OF DENMARK. :ENDIF :EOJ Looks good? It did to me, too, but whenever one of the FCOPY commands aborts, the FCOPY subsystem is exited and the next FCOPY line -- the second "FROM=;TO=" or the "EXIT" -- is caught by MPE. MPE sees this as an MPE command, notices that it doesn't start with a ":", and flushes the job. The ":CONTINUE" does prevent the error in :FCOPY from aborting the job stream, but it doesn't have any effect beyond the :FCOPY command, and the "command" that MPE thinks it sees (which is actually a leftover FCOPY subsystem command) makes the job stream die. What can I do? A: Your best solution would be to split the FCOPY call into several commands, to wit: :FCOPY FROM=MYFILE1;TO=MYFILE2;NEW :FCOPY FROM=MYFILE3;TO=MYFILE4;NEW Then, you could put a :CONTINUE before and an :IF after each of the :FCOPY commands, e.g. :CONTINUE :FCOPY FROM=MYFILE1;TO=MYFILE2;NEW :IF JCW<>0 THEN : TELLOP SOMETHING IS ROTTEN IN SOUTH DENMARK. :ELSE : CONTINUE : FCOPY FROM=MYFILE3;TO=MYFILE4;NEW : IF JCW<>0 THEN : TELLOP SOMETHING IS ROTTEN IN NORTH DENMARK. : ENDIF :ENDIF Note that the same problem could happen in some program other than FCOPY; in this case, this solution won't work, since most programs can't be invoked like FCOPY (":FCOPY fcopy-command"). The more general (albeit more cumbersome) solution is to have the job stream put all the input lines into a disc file (using :EDITOR or :FCOPY) and then run the program with $STDIN redirected to a disc file. Q: I have several production machines DSed to our development machine. How can I transfer data bases over the DSLINE to a remote CPU? A: With great difficulty. As I'm sure you've figured out, neither DSCOPY or even (yecch) FCOPY can handle IMAGE files; this is because they're privileged files, and require special shenanigans to open and access. Even ROBELLE's excellent SUPRTOOL doesn't allow copying databases (although it allows almost everything else). If you want to stick with HP utilities, the only thing you can do is :STORE the database to tape and :RESTORE it on the target machine. Granted, this is rather slow, requires operator intervention, and may cause substantial problems if the machines aren't in the same building, but that's all that HP gives you. Fortunately, MPEX/3000 is available from VESOFT. MPEX allows you to change a file's filecode; since a privileged file is identified by its negative filecode, you can convert it to a positive code, copy the file (now no longer privileged) and convert it back to a negative value on the target machine. Thus, say you want to copy the database MYDB across DSLINE DSDEV. You'd do the following: On the local machine: (sign on as a user with PM capability; otherwise, MPEX won't let you change the priv file's file code) :RUN MPEX.PUB.VESOFT %!ALTFILE MYDB,INTCODE=+400 << change from -400 to +400 >> %!ALTFILE MYDB##,INTCODE=+401 << change from -401 to +401 >> %DSLINE DSDEV %REMOTE HELLO USER.ACCT << now call DSCOPY on all the MYDB files >> %!USER DSCOPY.USER,MYDB@,MYDB@,DSDEV %!ALTFILE MYDB,INTCODE=-400 << change back to the right code >> %!ALTFILE MYDB##,INTCODE=-401 << change back to the right code >> %EXIT Now, on the remote machine: :RUN MPEX.PUB.VESOFT %!ALTFILE MYDB,INTCODE=-400 << change back to the right code >> %!ALTFILE MYDB##,INTCODE=-401 << change back to the right code >> %EXIT THE FILENAME OF THE DATABASE ON THE TARGET SYSTEM MUST BE THE SAME AS ON THE SOURCE SYSTEM. The group and account names may be different, but the filename must be the same. Yes, I know, this is a fairly messy way of doing things, but unfortunately this and :STOREing the database are the only real alternatives available. Q: We have a 2621P terminal as our system console. We'd like to let programs send special messages to the operator in which they turn the integral printer on, print the message, and then turn the integral printer off. However, when we just send the message using :TELLOP, all the escape sequences we use to manipulate the printer are stripped. What can we do? A: First, allow me to give you a bit of historical background. When the :TELL and :TELLOP commands were first built, they just took the message and printed it to the target terminal. In the days of the old dumb terminals, this was quite fine. Then, along comes the HP 264X series terminal with its potpourri of escape sequences. One of these escape sequences is "ESC d", which causes the terminal on which it is printed to send to the computer the current line of the display. So, some wise guy enters the following command: :TELL MANAGER.SYS <ESC>B ALTACCT DEV;CAP=IA,BA,...,PM,SM<ESC>d The message is printed on a terminal that's signed on as MANAGER.SYS, the "<ESC>B" makes the terminal go to a new line, the :ALTACCT command is displayed, and the "<ESC>d" tells the terminal to send the command to the computer. Simple and effective -- with one :TELL command, you can "enslave" any session and make it do your bidding. HP got wind of this, and realized that it had to do something about it. So, now the :TELL and :TELLOP commands (and PRINTOP intrinsic) strips out all escape sequences from the message except for a select few legal ones (e.g. "<esc>&d" which sets the terminal enhancements). Fortunately, what the security system taketh, privileged mode can give back. The :TELLOP command, after stripping the escape sequences, calls a system-internal, privileged, and VERY DANGEROUS IF YOU DON'T CALL IT RIGHT procedure called GENMSG. With PM, you can call GENMSG directly. Just compile the following procedure and add it to your system SL (or any SL with PM capability): $CONTROL NOSOURCE, USLINIT, SUBPROGRAM, SEGMENT=TELLOP'ESCAPE BEGIN PROCEDURE TELLOPESCAPE (BUFFER, LENGTH); BYTE ARRAY BUFFER; INTEGER LENGTH; OPTION PRIVILEGED; BEGIN << This procedure sends the message contained in BUFFER to the system console; BUFFER may contain escape sequences. LENGTH must be the length of the message in bytes, NO MORE THAN 255!!! GOD FORBID THAT YOU SHOULD PASS AN INVALID BUFFER ADDRESS OR AN INVALID LENGTH!!! >> BYTE ARRAY TEMP(0:255); INTEGER PROCEDURE GENMSG (NSET, NMSG, MASK, P1, P2, P3, P4, P5, DEST, REPLY, OFFSET, DSEG, CONTROL); VALUE NSET,NMSG,MASK,P1,P2,P3,P4,P5,DEST,REPLY,OFFSET,DSEG,CONTROL; INTEGER NSET, NMSG, DEST, DSEG; LOGICAL MASK, P1, P2, P3, P4, P5, REPLY, OFFSET, CONTROL; OPTION EXTERNAL, VARIABLE; MOVE TEMP:=BUFFER,(LENGTH); TEMP(LENGTH):=0; GENMSG (-1,@TEMP,<<mask>>,<<p1>>,<<p2>>,<<p3>>,<<p4>>,<<p5>>, 0<<dest=console>>); END; END. You can call this procedure from any program and pass to it the byte array containing the message to be printed (including any escape sequences you'd like) and the length of the message. Note, however, that if you make this procedure publicly available, people can pull the same stunt that made HP forbid sending escape sequences in the first place. If you're concerned about this, you might put checking code into the procedure, or put the procedure into a group or account SL rather than making it available to everyone by putting it into SL.PUB.SYS. Q: How can a program retrieve the INFO= string passed on the :RUN command? A: Well, first of all, if you're a PASCAL user, you've got no problem. Just say: PROGRAM TEST (INPUT, OUTPUT, PARM, INFO); VAR PARM: -32768..32767; INFO: STRING[256]; BEGIN WRITELN (PARM); WRITELN (INFO); END. The variable "PARM" will contain your ;PARM= value; "INFO" will contain the ;INFO= string. This is what I call an easy-to-use interface. If you're a FORTRAN or COBOL (or, to some extent, SPL) user, you have no such luck. Instead, you have to use the following procedure: $CONTROL NOSOURCE, USLINIT, SUBPROGRAM, SEGMENT=GETPARMINFO BEGIN PROCEDURE GETPARMINFO (PARM, INFO, INFO'LEN); INTEGER PARM; BYTE ARRAY INFO; INTEGER INFO'LEN; BEGIN INTRINSIC TERMINATE; INTEGER POINTER QPTR; INTEGER Q=Q; BYTE ARRAY RELATIVE'DB'(*)=DB+0; DEFINE QPTR'SEGMENT = QPTR(-1).(8:8) #, QPTR'DELTAQ = QPTR(0) #; @QPTR:=@Q; WHILE QPTR'SEGMENT<>(@TERMINATE).(8:8) DO @QPTR:=@QPTR(-QPTR'DELTAQ); PARM:=QPTR(-4); INFO'LEN:=QPTR(-6); MOVE INFO:=RELATIVE'DB'(QPTR(-5)),(INFO'LEN); END; END. Add this procedure to your SL, and call it from your program. It will return to you the ;PARM=, the ;INFO= string, and the length (in bytes) of the ;INFO= string. Be sure that the buffer you pass to hold the ;INFO= string is actually big enough to fit all of it. How does all this work internally? Well, the PARM= parameter, the byte address of the INFO= string, and the length of the INFO= string are stored at locations Q-4, Q-5, and Q-6 (respectively) in your stack. However, as you call procedures in your program, the Q register gets moved up (to make room for local variables, procedure parameters, etc.). The data still remains at addresses Q-4 through Q-6, but relative to the INITIAL, not the CURRENT, value of the Q register. Fortunately, every time you call a procedure, a so-called "stack marker" is put on the stack which links the new procedure's Q register to the previous procedure's Q register. This procedure goes back through the stack markers until it finds the very lowest one, the one that resides at the place where Q pointed to when the program was first entered. At locations -4, -5, and -6 relative to this, we find the ;PARM= parameter and the ;INFO= address and length (which was what we wanted). How do we know that this is the lowest stack marker? Because it points into the so-called "MORGUE" segment in the system SL, in which the TERMINATE intrinsic happens to also reside. So, when we find a stack marker that points to the same segment in which TERMINATE is located (=(@TERMINATE).(8:8)), we know that we're done. Q: Can you explain what happens when I run QUERY.PUB.SYS from inside SPOOK.PUB.SYS? No output goes to the screen, although QUERY seems to be accepting input, and writing stuff to a file it builds called QSOUT. In a similar vein, when I try to run a compiled BASIC program within SPOOK, it fails with a CHAIN ERROR. Can you explain? A: It is to the credit of the author of SPOOK that SPOOK has a RUN command in the first place. Note that until TDP, no other HP product (except perhaps BASIC) allowed one to run other programs within it (SPOOK, incidentally, started out as an SE-written program and only later became an HP-supported utility). Apparently, one of the reasons that SPOOK's author put the >RUN command in in the first place was to allow SPOOK to be run from within itself! Presumably, this could be useful for being able to look at two spool files at a time; frankly, I don't see the need for it, but all the signs point to the author's desire to be able to run SPOOK from itself. Try going into SPOOK and saying > RUN SPOOK.PUB.SYS A new copy of SPOOK will be launched, AND IT WILL PROMPT YOU NOT WITH "> ", BUT WITH ">(1)"! If you run SPOOK again from within the newly-created process, the newest process will prompt you with ">(2)", and so on. If you run SPOOK from itself 10 times, instead of getting the ">(10)" that you'd expect, you'll get ">(:)". Apparently, SPOOK's author decided that it was important to indicate which copy of SPOOK was which, so SPOOK always run its sons with ;PARM=49 (which is the ASCII equivalent of "1"). If a copy of SPOOK was already with a PARM=, it creates all its sons with a PARM= value 1 greater (i.e. 50, 51, etc.). The prompt is then set to ">(x)" where "x" is the ASCII character corresponding to the PARM= value. The bottom line is that when you run SPOOK in the default state (with ;PARM=0), it will run all its sons with PARM=49. This is usually OK, except for those programs -- like QUERY and BASIC -- that act differently when their ;PARM=0; in those cases, life gets difficult. Unfortunately, on SPOOK's >RUN, you can't specify a different ;PARM= value; you're pretty much stuck with what SPOOK gives you. You can, however, use one or more of the following workarounds: * Don't run QUERY or BASIC compiled programs from SPOOK. * Alternatively, if you're an MPEX customer, you can "hook" SPOOK to accept any line starting with a "%" as an MPEX command, which may be a %RUN (with arbitrary parameters), %PREP, UDC, or anything else. This kind of "hooking" can be done to any program, and MPEX users regularly hook EDITOR, TDP, QUAD, LISTDIR5, RJE, etc. to recognize MPEX commands. * In the case of QUERY, QUERY uses ;PARM=<any odd number> to indicate that output should be sent to the file called QSOUT. If you issue a file equation :FILE QSOUT=$STDLIST then even if QUERY is run with PARM=49, it'll still output stuff to $STDLIST (even though it think it's going to QSOUT). * Finally, you can :RUN SPOOK.PUB.SYS;PARM=-1 to start with. This will cause SPOOK to prompt you with ">()" (since ASCII character -1 is unprintable); all RUNs from within SPOOK will be done with a ;PARM= value one greater than the one which was passed to SPOOK -- in this case, -1+1, or 0! This way, you fool SPOOK into running all its sons with ;PARM=0, just like it should have in the first place. This, I think, can serve as a valuable lesson: BEWARE OF GALLOPING NIFTINESS! Before adding more and more baroque features to a program, consider that perhaps the simplest solution -- run everything with ;PARM=0, even if it means that if you run SPOOK from SPOOK, the prompt will still be "> " -- may be the easiest one to live with. Q: I want to access a KSAM data file as a normal MPE file so that I can do a quick sequential read against it. I have tried :FILE equations and several FOPEN parameters without success -- what can I do to view the data file independently from the key file? A: The :FILE equation has a ;COPY parameter on it, which corresponds to aoptions bit .(3:1). When this bit or :FILE parameter is set, any special file -- KSAM, MSG, etc. -- looks like a plain vanilla MPE file. A KSAM file is read chronologically, ignoring key sequence (and including deleted records!); a MSG file is read non-destructively, with no waiting at end of file. Whenever you want to ignore the underlying structure of a special file, this is the mode for you. Watch out, though -- some of the structure that you're ignoring may be important. KSAM files, for instance, mark their deleted records using a -1 in the first word; normally, these records won't be seen when you read the file, but if you open the file with ;COPY access, you will read them just as you would normal records. You should explicitly check for a -1 in the first word, and ignore all those records. Incidentally, to read a KSAM file with ;NOBUF, you MUST also use ;COPY -- otherwise, you'll get one record at a time, just as you would in buffered mode. On the other hand, it's perfectly OK to read a KSAM with ;COPY but without ;NOBUF. Q: What is there about the HELLO command that prevents me from REDOing it? Is it also the same reason why HELLO can not be a command in a UDC? A: Most MPE commands can only be executed within a session, where there's a Command Interpreter process to deal with them. :HELLO, :JOB, and a few others, however, must be executable when no session yet exists; they are executed by DEVREC (DEVice RECognition), and don't support all the features that are present with ordinary Command Interpreter commands. In :REDO, for instance, the CI inputs the changes to the command, and when the user is done, executes the new command. However, since :HELLO can only be executed by DEVREC (not by the CI), the CI rejects the command, and prints JOB/HELLO/DATA WILL NOT EXECUTE PROPERLY IN THIS CONTEXT, PLEASE REENTER COMPLETE COMMAND (CIWARN 1684). As you guessed, this is also the reason why :HELLO doesn't work from UDCs. Of course, this is rationalization more than reason; HP could have, without great grief, implemented :JOB, :HELLO, and :DATA so that they could be executable from :REDO and UDCs. However, the payoff for this would have been relatively small, and so the existing division between DEVREC commands (:JOB, :HELLO, :DATA) and CI commands (everything else) was let stand. Q: I need to get 20K sectors of contiguous disc space on a system disc drive. I used MPEX to move several files off that disc, but even though I was left with 25 K sectors on the disk, they were not contiguous. How can I condense this disc space without a RELOAD? A: The cure for all your woes is VINIT. By saying :VINIT >COND 1 you can condense space on disc drive #1, which should hopefully give you the contiguous space you need. This is quite a bit faster than a RELOAD, although while it's running, the other users on the system won't be able to get much work done. Unfortunately, this is also not as thorough as a RELOAD, since VINIT balks at condensing the smaller free space chunks and will typically only merge those chunks >100 to 1000 sectors -- the small fry, 1-100 sector holes, will be untouched. Still, it's quick enough that you can try it, and if it doesn't give you the desired results, then you can RELOAD. Q: I'd like to let one of my COBOL subroutines save data from one call to another, but I don't want to use a huge chunk of working-storage or use an extra data segment. How can I do this? I've heard that the DL-DB area of the stack can be used for this purpose, but how can I access it from COBOL? A: There are two answers to your problem, the simple one and the complicated one. The simple one is to use "$CONTROL SUBPROGRAM" instead of "$CONTROL DYNAMIC" at the beginning of your procedure source file (see Appendix A of the COBOL Reference Manual). This makes all your variables "static", i.e. allocated relative to DB and thus retained from call to call, as opposed to "dynamic", which means that they're deallocated every time the procedure exits. However, since Connie Wright wants the Q&A column to be of decent length, I'll also give you the complicated answer. God knows, it might even come in handy if the $CONTROL SUBPROGRAM solution isn't sufficient (perhaps if you want to put the procedure in an SL, where "$CONTROL SUBPROGRAM" procedures aren't allowed). The stack of every process is layed out rather like this: ----------------------------------------------------------------- ... | global variables | subroutine-local variables | ... ----------------------------------------------------------------- <- DL DB Qi S Z -> Between DB and Qi (the initial location of the Q register), you'd keep your global variables; between Qi and S, the procedure-local variables. This requirement is enforced by the HP3000's architecture because every time a procedure is exited, all the local variables that were allocated for it above its Q register have their space deallocated. Next time the procedure is called, the local variables will be reallocated at new addresses, and will of course have new values. Everything below Qi -- including DB-Qi and DL-DB -- is not automatically changed by procedure calls and returns. The compilers allocate global variables between DB and Qi, so if you try to use that space, you'll interfere with them, but you can safely (fairly safely) put stuff between DL and DB, and unless you explicitly change it, it'll remain the same no matter how many procedures you call or exit. What IS the DL-DB area? It is just a part of the stack, containing memory locations belonging to your process (like DB-Qi or Qi-S). There are several rules, however, that you must follow to access this area: * The locations in this area have NEGATIVE addresses (since they're below DB and all stack addresses are DB-relative). This means that they can only be accessed directly using SPL, and you have to write special SPL interface procedures to access them from other languages. * By default, any program has less than 100 words of space available between DL and DB. If you want more (as apparently you do), you can run your program with ;DL=nnn to ensure that at least nnn words of DL-DB space will be available. Even better, you can use the DLSIZE intrinsic to dynamically expand and contract your DL-DB space: CALL INTRINSIC "DLSIZE" USING -1280, GIVING DLSIZE-GRANTED. I usually use DLSIZE instead of running my program with ;DL=, not just because it allows me to dynamically change the amount of space I have allocated, but also because it saves me embarrassment when I forget to :PREP my program with ;DL=. * Finally, although the system left the entire DL-DB area for "user applications", from MPE's point of view V/3000 and the like are also "user applications", and some of them are quite liberal with their DL-DB space usage. V/3000 is the big culprit, so I'd suggest that you avoid using the DL-DB area in programs that use V/3000. If you don't use V/3000, you should still stay clear of the area between about DL-16 and DL-1, which is used by SORT/MERGE, COBOL, and the FORTRAN formatter. So the point is that, once you've allocated enough space with ;DL= or DLSIZE, and have made sure that you're not colliding with anybody else (like V/3000), all you need to do is write a few small SPL procedures: $CONTROL NOSOURCE, SEGMENT=ADDR'WALDOES, SUBPROGRAM, USLINIT BEGIN PROCEDURE MOVEFROMADDR (BUFF, ADDR, LEN); VALUE ADDR; VALUE LEN; INTEGER ARRAY BUFF; INTEGER ADDR; INTEGER LEN; BEGIN INTEGER ARRAY RELATIVE'DB(*)=DB+0; MOVE BUFF:=RELATIVE'DB(ADDR),(LEN); END; PROCEDURE MOVETOADDR (ADDR, BUFF, LEN); VALUE ADDR; VALUE LEN; INTEGER ADDR; INTEGER ARRAY BUFF; INTEGER LEN; BEGIN INTEGER ARRAY RELATIVE'DB(*)=DB+0; MOVE RELATIVE'DB(ADDR):=BUFF,(LEN); END; END. Saying CALL "MOVETOADDR" USING -1023, MY-STUFF, 100. will move the first 100 words of MY-STUFF to the 100 locations starting with -1023; similarly, CALL "MOVEFROMADDR" USING MY-STUFF, -1023, 100. will move the 100 words starting with -1023 to MY-STUFF. In the above examples, "-1023" is the address of the DL-DB chunk you're using. It's a negative number like all DL-DB addresses, and it could either be a constant (if you always know that words -1023 through -924 are the ones that you'll use), or a variable. As you may have noticed, MOVETOADDR and MOVEFROMADDR move to and from any arbitrary address in the stack, with no concern as to whether it's between DL and DB or not. They are simply interfaces to SPL's arbitrary address handling capability; the COBOL program assigns the addresses, allocates the space, etc. Many such things that can "only be done in SPL" may easily be implemented in other languages by writing very simple SPL procedures that can be called from COBOL, FORTRAN, PASCAL, etc. programs. Q: How can I determine all the UDC files :SETCATALOGed on my system? Also, can I group UDC files together to improve performance? A: Starting with T-MIT (I believe -- it may have been T-delta or something like that), HP allows a user with SM capability to say :SHOWCATALOG ;USER=@.@ to show all the UDCs files currently set on the system level. That means all the files set using ":SETCATALOG ;SYSTEM", something that any user could always do without SM using the :SHOWCATALOG command. How incredibly useful. What you want -- and quite reasonably, I might add -- can NOT easily be done with any HP command. It should be, but it can't. There's no justice in the world. Fortunately, this information is not so hard for you to get at yourself. All the information on who has what UDCs set is kept in COMMAND.PUB.SYS, an ordinary file with a fairly simple format (described, in its usual incredibly readable style, by the System Tables Manual in Chapter 15). Simply put, * Record 0 is a header. * Any other record with a 1 in its word #1 is a "user record", which contains the user and account for which a UDC file (or files) is set. The user name starts in word 2, the account name in 6; account-level UDCs have an "@" in the user field, and system-level UDCs have an "@" in both the user and account. * Word #0 of any "user record" contains the record number of the first "file record" for this user. Each file record's word #0 contains the record number of the next file record belonging to this user, until the user's last file record, which has a 0 in its word #0. Each file record contains the filename starting with word 2. So, all you have to do is read through COMMAND.PUB.SYS, finding each user record and traversing its linked list of file records. In fact, that's exactly what the following quickie SPL program does: $CONTROL NOSOURCE, USLINIT << )C( COPYWRONG 1985 BY VESOFT. NO RIGHTS RESERVED. >> BEGIN INTRINSIC FOPEN, FREADDIR, PRINT; INTEGER ARRAY REC(0:19); INTEGER FNUM; BYTE ARRAY FILE(0:35):="COMMAND.PUB.SYS "; DOUBLE RECNUM; FNUM:=FOPEN (FILE, 1); RECNUM:=1D; << 0th record is just a header >> FREADDIR (FNUM, REC, 20, RECNUM); WHILE = DO BEGIN IF REC(1)=1 THEN << user record >> BEGIN PRINT (REC(2), -16, 0); WHILE REC(0)<>0 DO BEGIN FREADDIR (FNUM, REC, 20, DOUBLE(REC(0))); PRINT (REC(2), -36, 0); END; PRINT (REC, 0, 0); << blank line >> END; RECNUM:=RECNUM+1D; FREADDIR (FNUM, REC, 20, RECNUM); END; END. For brevity's sake, I've omitted much of the prettifying, so all you'll see is the name of each user who has UDCs set, followed by the names of all his files, followed by a blank line. Clean it up as you will, and put it in some revered spot on your system (note: this printout will show the lockwords for any UDC files which were :SETCATALOGed filename/lockword). It's likely to come in handy. About performance: Run-time performance (i.e. how long it takes MPE to recognize and parse your UDC invocation) is affected solely by the number of UDCs (not UDC files) a particular user has set; in any event, UDC lookup and expansion are usually very fast. Performance at logon time can be rather poor because MPE must open and read every UDC file set up on the user's behalf. The order of entries in COMMAND.PUB.SYS doesn't matter; what does matter is the number of UDC files (very important, since there's an FOPEN for each one and FOPENs ain't cheap), the combined size of the UDC files, and the blocking factor of each. First, try merging several UDC files into one; after that, increase the blocking factor of each UDC file to minimize the number of I/Os necessary to read it. "HONEST WILFRED"'s HP3000 CORRESPONDENCE SCHOOL, INSTALLMENT #34: This month is our "breadth requirements" exam. Please solve the following problems: PHILOSOPHY: "The HP3000 does not think, therefore it does not exist." Do you agree or disagree? Why? MATHEMATICS: Construct an HP3000 with straight edge and compass. NUMEROLOGY: 3000 is four times 666, plus twice 13 squared, minus 2. From this information, calculate the date of the end of the world. For extra credit, calculate the date of the obsolescence of the 3000 line. Show all work. You have 45 minutes. Send your answers, together with this month's check for $342.00, to Honest Wilfred's HP3000 Correspondence School 459 Broadway Truth or Consequences, NM 97314 Q: We do our system cleanup -- things like a VINIT >CONDense and such -- at night without operator assistance. However, some people sometimes leave their sessions logged on overnight, and sometimes jobs are started up in the evening that are still running when we want to do the cleanup. We'd like to be able to automatically abort all the jobs and sessions in the system (except, of course, the job doing the aborting). How can we do this? A: What we would really like to have -- what would make the problem trivial to solve -- is a command of the form :ABORTJOB @.@ Try finding that one in your reference manual. The sad fact, of course, is that no such a command exists. However, we do have two commands that each do about half of the task: * :SHOWJOB, which finds all the jobs in the system. * :ABORTJOB, which aborts a single job. The trick is to take the output of the :SHOWJOB command and feed it to the :ABORTJOB command, so all the jobs shown are aborted. Without much further ado, here's the job stream we want: !JOB ABORTALL,MANAGER.SYS;OUTCLASS=,1 !COMMENT by Vladimir Volokh of VESOFT, Inc. !FILE JOBLIST;REC=-80;NOCCTL;TEMP !SHOWJOB;*JOBLIST !EDITOR TEXT JOBLIST DELETE 1/3 DELETE LAST-6/LAST FIND FIRST WHILE FLAG DELETE "MANAGER.SYS" CHANGE 8/80,"",ALL CHANGE 1,":ABORTJOB ",ALL KEEP $NEWPASS,UNN USE $OLDPASS EXIT !EOJ A brief explanation: * First we do a :SHOWJOB of all the jobs in the system to a temporary disc file (note that this redirection of :SHOWJOB output to a disc file might not be documented -- it certainly isn't documented in :HELP, nor is it mentioned in some of the older manuals). * Then we enter editor and massage the file so that it contains only the lines pertaining to the actual jobs (all the lines except the first 3 and the last 7). Then, we exclude all the jobs signed on as MANAGER.SYS, since we don't want to abort our system cleanup job streams (like ABORTALL itself or the job that streamed it), and they are presumably signed on as MANAGER.SYS. Of course, you can easily change this to any other user name. * Now, we strip all the data from the lines except for the job number, and then we insert an :ABORTJOB in front of the number. The file now looks like: :ABORTJOB #S127 :ABORTJOB #S129 :ABORTJOB #J31 :ABORTJOB #S105 :ABORTJOB #J33 * Finally, we keep this as a disc file (in our case, $NEWPASS), and then we /USE this file, causing each of its commands to be executed as an EDITOR command. Since ":ABORTJOB" is a valid EDITOR command -- it just makes the EDITOR call the COMMAND intrinsic -- all these commands get executed! * Note that in order for this job stream to work properly, the :ABORTJOB command must be :ALLOWed to MANAGER.SYS. This can be done via the :ALLOW command or by use of the ALLOWME program in the contributed library (see the answer to the previous question for more information). And there's your answer. Fairly simple, no need to go into privileged mode, no need to even write a special program (other than the job stream itself). A perfect example of "MPE PROGRAMMING" -- performing complicated system programming tasks without having to write a special program in SPL or some such language. Incidentally, for more information on MPE Programming, you can read my "MPE Programming" article. Q: Sometimes we have a program file and aren't sure which version of source code it came from. Usually, we have a strong suspicion which one it is, so we tried to simply recompile the suspected source file and then use :FCOPY ;COMPARE to compare the compiled source with the program file. Unfortunately, FCOPY gave us hundreds of discrepancies; in fact, we found that the program file changed every time it was run! What can we do? A: As you pointed out, every time a program is run, various parts of it -- record 0, the Segment Transfer Tables (STTs) in each segment, etc. -- are changed. FCOPY is therefore a dead end, since it's virtually impossible to tell a legitimate discrepancy from a difference caused by the loader. However, there are some things you can do. First of all, you can of course :LISTF both program files -- if they are of different sizes, they're clearly substantively different. Of course, if they're of the same size, you can't be certain that they are actually the same. Beyond that, your best solution is to have some kind of version numbering system. If you can be sure to increment a version number variable in your source every time you make a change, and then print the version number whenever you run the program, you can very easily tell whether two program versions are the same just by looking at their version numbers. The only problem with this is that it's rather easy to forget to change the version number. If you think this will happen often, you could try some more automatic but less reliable approaches. For instance, you could simply compare the creation dates of the program file and the source file; since whenever you /KEEP a file in EDITOR, it's re-created, thus updating its creation date, it's a pretty good bet that if the two creation dates are equal, you're dealing with the same version. Of course, this approach is hardly foolproof -- either file might have been copied, thus changing its creation date, or two changes might have been made on the same day. Unfortunately, this is as precise as you're going to get without some kind of version numbering system in the source code. Q: One of my programs keeps aborting with a FSERR 74 ("No room left in stack segment for another file entry") on an FOPEN. True, I do have a procedure that allocates a very large array on the stack -- I run my program with ;MAXDATA=30000 -- but I don't do the FOPEN at the time I'm in the procedure, so at FOPEN time my stack can't be larger than about 15K. What am I doing wrong? A: To answer this question, I have to digress for a moment and discuss the structure of your stack. All of the variables and arrays you declare in your program -- local or global -- are put onto your stack, which is stored by MPE in a single data segment. Now, you don't view your stack as a data segment, since you don't need to call DMOVIN or DMOVOUT to access it; however, deep down inside the system does, and places on the stack a fundamental restriction common to all data segments -- no data segment may ever be more than 32K words long. Now, your stack is actually partitioned into several pieces, each piece pointed to by a machine register: High memory Z > --------------------------------------------- ^ | unused space | ^ S > --------------------------------------------- ^ | operands of machine instructions | ^ | and local variables of the currently | ^ | executing procedure | ^ Q > --------------------------------------------- ^ | local variables of other procedures | ^ | and stack markers indicating calls from | ^ | one procedure to another | ^ Qi > --------------------------------------------- positive | global variables | addresses DB > --------------------------------------------- | "DB negative area," accessible only by | negative | SPL procedures (like V/3000) -- usable | addresses | for global storage | v DL > --------------------------------------------- v | The Nether Regions, where mortals may | v | not stray and non-privileged accessors | v | are punished with bounds violations | v --------------------------------------------- v Low memory (The above picture reproduced with permission from "The Secrets of Systems Tables... Revealed!" by VESOFT. Said permission was not especially difficult to obtain.) Now your stack contains all of this stuff, from the bottom of the Nether Regions to your Z register. The Nether Regions (the DL negative area, also knows as the PCBX) is where the file system allocates miscellaneous information about your file; when you open enough files, the initial space allocated below DL is exhausted, and the system has to allocate some more. Now, if this would make the total size of the stack data segment larger than 32K, you get an FSERR 74. The problem is that the data segment size is not measured from the bottom of the PCBX to the S register, but rather to the Z register. Your S register points to the current top of your stack, so if you're using 15K, your S register value is 15K; however, your Z register points to the highest place that the S register has ever been! Thus, say your stack size starts out at 10K. Your S points to 10K, and your Z points to about 12K (since the system allocates about 2K of overhead between S and Z). Now, you call your procedure which allocates an 18K local variable; now your S is at 28K and Z is at 30K. When you exit the procedure, the S is reset to 10K, but the Z stays at 30K! This leaves you only 2K for the PCBX, even though you have 20K of unused space between S and Z. The solution? Well, I think that your best solution is simply to call the ZSIZE intrinsic, passing to it the parameter 0, before calling FOPEN. This call frees any space you may have between S and Z, thus leaving the maximum possible space for the file system to work with. But, you might say, if the file system allocates an extra 1K in the DL negative area, this will decrease the maximum Z value to 29K, and so when I next try to allocate that 18K array, I'll get a stack overflow! Right? Wrong. It turns out that the highest value I could get Z to take was 30848 (use this number for comparison, your mileage may vary with your MPE version). Thus, if you allocate enough space for S to be pushed up to 28K, Z will be set to 30848; however, if you push S all the way up to 30K, Z will be left at the same value of 30848 without any adverse effects. Thus, you can contract the S-Z area by calling "ZSIZE (0)", call FOPEN, have it allocate a couple of hundred extra words in the DL negative area, and still be able to push S and Z up by 18K with no problems! So, doing a "ZSIZE (0)" before an FOPEN will probably solve your problem (I do this in my MPEX program all the time). However, if it doesn't -- if you are really using every word of stack space and can't fit both your own data and the file system's file information into one 32K stack -- there is an alternative; you may run your program with ;NOCB, which causes most of the file system information to be allocated in a separate data segment. The reason why I didn't suggest this first was that this slightly slows down file system accesses; furthermore, if your program doesn't run without ;NOCB, you can bet that half the time the user will forget to specify ;NOCB and will get an error. I think that the ZSIZE call, if it works, is the cleaner solution. Q: What effect does virtual memory size have on the performance of my system? Can I configure it too large? Too small? A: None. No. Yes. What? You want to know more? I gave you the answers, didn't I? Oh, all right. If you have too little main memory, this will undoubtedly affect your system performance because there'll be a lot of swapping. However, let's pretend that there was no virtual memory -- that if you ran out of main memory, you'd simply get an error message (instead of getting an unused segment swapped out to disk). In that case, the size of main memory would actually have no effect on the performance of your system, if all you mean by performance is the speed at which the system runs. If you configure your main memory too small, you'll simply get an error message when you try to overflow it; however, whatever can run in the reduced memory will run as fast as it would in a configuration with a lot of memory. That's exactly what happens with virtual memory. If virtual memory gets full, you'll just get an error message; then, you can reconfigure your system to have more virtual memory, and that's that. If you want to check how much virtual memory you're actually using, you can run the contributed utility TUNER. So, as long as you don't run out of virtual memory, changing its size will not affect your system performance one iota. The only problem with making it too large is that you waste some disc space (which is fairly cheap anyway). The System Operation and Resource Management Manual suggests that you determine the amount of virtual memory you will need as follows: estimate the average number of concurrent users, the average stack size, the average number of buffered files open for every user, and the number of users who will be simultaneously running programs; then use these figures in conjunction with the values listed below to calculate the amount of disc space to reserve for virtual memory: * 32 sectors for each Command Interpreter stack * 4 sectors for every open buffered file * 16 sectors for the system area in the user's stack, plus 4 sectors for every 512 words in the DL/Z area of the stack * 40 sectors for each program being loaded in the system The amount of virtual memory calculated by the above is adequate for the overwhelming majority of systems. If you actually run out of it, just increase it. Q: We would like to know how the CALENDAR and FMTCALENDAR intrinsics are going to cope with dates beyond 31 December 1999. We do not think it premature to seek clarification on this topic, because each user of the HP3000 system will have programs and procedures that compare dates and assume that the smaller number is the earlier date. With the CALENDAR intrinsic, bits 0 to 6 represent the year of the century and bits 7 to 15 represent the day of the year. For non-computer people, we usually describe this dating convention as 512 * (year of century) + (day of year). On 3 January 2000 if we want a SYSDUMP to include all files that have been accessed since 28 December 1999, so we enter 12/28/99 as the response to "ENTER DUMP DATE?". Now, how can the computer recognize that files accessed on 1, 2, or 3 January with an access date of %1, %2, or %3 are actually after year 99 day 362 (which is represented as 512 * 99 + 362 = %143552). Somehow, %1 will have to be recognized as subsequent to or effectively a larger number than %143552. To demonstrate the present inability of MPE to cope with this, users can on their next system restart enter a system date of 12/31/99 and a time of 23:58 and do a :SHOWTIME after each elapsed minute to see what happens when the century changes. We have submitted this problem on a Service Request and it has been assigned a number 4700094458 but we are not aware of any progress as it has not yet appeared on the System Status Bulletin. A: You raise a very interesting question that is rapidly becoming more and more relevant to system design. The actual CALENDAR intrinsic will not be the first thing to fail come the new century (new millennium!). There is room in there for up to 128 years, so HP should be OK all the way until 2027, as long as it recognizes that 12/31/03 is actually 31 December 2003, not 1903. FMTCALENDAR (and DATELINE, :SHOWTIME, etc.) will all be in a bit more trouble. If you're too impatient to see for yourself by resetting your system clock, I'll tell you that on 1 January 2000 00:00, the :SHOWTIME will read SUN, JAN 1, 19:0, 12:00 AM The ":" comes because it's the next ASCII character after "9"; the formatted algorithm uses a simple-minded approach which say that the decades digit is always ("0"+YEAR/10) and the years digit is ("0"+YEAR MOD 10). However, this will only hurt you if you rely on FMTCALENDAR or DATELINE in your program. After all, if all you care about is :SHOWTIMEs, when will you forget what decade you're in? The big problem is that ALMOST ALL USER PROGRAMS RELY ON 6-DIGIT DATES. It's not just that the HP can't handle the 21st century; chances are that your payroll, accounts payable, general ledger -- all your programs will believe that something done on 01/01/00 was long before 12/31/99. That is the thing people should start worrying about soon. The good news, of course, is that things aren't pressing just yet. It's unlike that the 3000 line will survive into the 21st century, and the few of you who might have one stashed in your attic will be laughed at for using "those old dinosaurs that couldn't even talk." The designers of the 3000 are probably not too worried about this problem right now. You -- the working programmers of the world -- should however start getting concerned, if not now, about 5 years from now. That's when programs that you write will begin to have a reasonable chance of surviving into the 21st century. At least, they'll be likely to at some time need to work with 21st century dates (as in "this loan is due 10 years from now in 2003"). You probably do not want to have your comparison routines fail and notify the debtor that his loan is 91 years overdue. Actually, there exists right now a small (but ever-growing) number of programs that at some time will have to concern them with this. I suggest that anyone designing new application systems -- and, after all, your system will probably outgrow the current computer you're on, migrating to Spectrum and god-knows-what-else -- plan for 8-digit dates (MM/DD/YYYY). This will waste 2 bytes per date, but in these days of 400 Mb discs, you can afford this relatively easily. Putting on my Nostradamus (or is it Cassandra?) hat, I predict that most people will not do this Until It's Too Late, even though far more weighty voices than mine will start suggesting this fairly soon. Then, about 1997 or so, they will See The Light, say Oh, My God, What Do We Do Now, and hire great batches of programmers to correct all of their old programs. This will rescue millions of programmers from dismal fates as grocery clerks, sanitation engineers, or data processing managers, and will be hailed far and wide as the Boom of '97. Congress, seeing the stimulating effect of this on the economy -- all those new jobs! -- will find this all A Very Good Thing, and will pass the National Date Standards Act of 1997, which will mandate that henceforth all dates must be stored in 5 digits, thus necessitating modification every 10 years. Senator Prescott of Massachusetts will propose an amendment that requires storage in 4 digits, but this will be voted down 64-29 (7 abstaining). See, they never listen to me. Oh, your Service Request? I'm sorry to say that even as I write this, it has inadvertently been filed in a small, cylindrical file in Building 43U, and will soon meet an ignoble fate at the hands of Rover, who frequents the garbage cans thereabouts. A future archaeologist will uncover shreds of it right next to the last three Service Requests that I've sent in, and will write a doctoral thesis. Any other questions? Q: Can you explain what happens when I run QUERY.PUB.SYS from inside SPOOK.PUB.SYS? No output goes to the screen, although QUERY seems to be accepting input, and writing stuff to a file it builds called QSOUT. In a similar vein, when I try to run a compiled BASIC program within SPOOK, it fails with a CHAIN ERROR. Can you explain? A: It is to the credit of the author of SPOOK that SPOOK has a RUN command in the first place. Note that until TDP, no other HP product (except perhaps BASIC) allowed one to run other programs within it (SPOOK, incidentally, started out as an SE-written program and only later became an HP-supported utility). Apparently, one of the reasons that SPOOK's author put the >RUN command in in the first place was to allow SPOOK to be run from within itself! Presumably, this could be useful for being able to look at two spool files at a time; frankly, I don't see the need for it, but all the signs point to the author's desire to be able to run SPOOK from itself. Try going into SPOOK and saying > RUN SPOOK.PUB.SYS A new copy of SPOOK will be launched, AND IT WILL PROMPT YOU NOT WITH "> ", BUT WITH ">(1)"! If you run SPOOK again from within the newly-created process, the newest process will prompt you with ">(2)", and so on. If you run SPOOK from itself 10 times, instead of getting the ">(10)" that you'd expect, you'll get ">(:)". Apparently, SPOOK's author decided that it was important to indicate which copy of SPOOK was which, so SPOOK always run its sons with ;PARM=49 (which is the ASCII equivalent of "1"). If a copy of SPOOK was already with a PARM=, it creates all its sons with a PARM= value 1 greater (i.e. 50, 51, etc.). The prompt is then set to ">(x)" where "x" is the ASCII character corresponding to the PARM= value. The bottom line is that when you run SPOOK in the default state (with ;PARM=0), it will run all its sons with PARM=49. This is usually OK, except for those programs -- like QUERY and BASIC -- that act differently when their ;PARM=0; in those cases, life gets difficult. Unfortunately, on SPOOK's >RUN, you can't specify a different ;PARM= value; you're pretty much stuck with what SPOOK gives you. You can, however, use one or more of the following workarounds: * Don't run QUERY or BASIC compiled programs from SPOOK. * Alternatively, if you're an MPEX customer, you can "hook" SPOOK to accept any line starting with a "%" as an MPEX command, which may be a %RUN (with arbitrary parameters), %PREP, UDC, or anything else. This kind of "hooking" can be done to any program, and MPEX users regularly hook EDITOR, TDP, QUAD, LISTDIR5, RJE, etc. to recognize MPEX commands. * In the case of QUERY, QUERY uses ;PARM=<any odd number> to indicate that output should be sent to the file called QSOUT. If you issue a file equation :FILE QSOUT=$STDLIST then even if QUERY is run with PARM=49, it'll still output stuff to $STDLIST (even though it think it's going to QSOUT). * Finally, you can :RUN SPOOK.PUB.SYS;PARM=-1 to start with. This will cause SPOOK to prompt you with ">()" (since ASCII character -1 is unprintable); all RUNs from within SPOOK will be done with a ;PARM= value one greater than the one which was passed to SPOOK -- in this case, -1+1, or 0! This way, you fool SPOOK into running all its sons with ;PARM=0, just like it should have in the first place. This, I think, can serve as a valuable lesson: BEWARE OF GALLOPING NIFTINESS! Before adding more and more baroque features to a program, consider that perhaps the simplest solution -- run everything with ;PARM=0, even if it means that if you run SPOOK from SPOOK, the prompt will still be "> " -- may be the easiest one to live with. Q: I'd like to let one of my COBOL subroutines save data from one call to another, but I don't want to use a huge chunk of working-storage or use an extra data segment. How can I do this? I've heard that the DL-DB area of the stack can be used for this purpose, but how can I access it from COBOL? A: There are two answers to your problem, the simple one and the complicated one. The simple one is to use "$CONTROL SUBPROGRAM" instead of "$CONTROL DYNAMIC" at the beginning of your procedure source file (see Appendix A of the COBOL Reference Manual). This makes all your variables "static", i.e. allocated relative to DB and thus retained from call to call, as opposed to "dynamic", which means that they're deallocated every time the procedure exits. However, since Connie Wright wants the Q&A column to be of decent length, I'll also give you the complicated answer. God knows, it might even come in handy if the $CONTROL SUBPROGRAM solution isn't sufficient (perhaps if you want to put the procedure in an SL, where "$CONTROL SUBPROGRAM" procedures aren't allowed). The stack of every process is laid out rather like this: ----------------------------------------------------------------- ... | global variables | subroutine-local variables | ... ----------------------------------------------------------------- <- DL DB Qi S Z -> Between DB and Qi (the initial location of the Q register), you'd keep your global variables; between Qi and S, the procedure-local variables. This requirement is enforced by the HP3000's architecture because every time a procedure is exited, all the local variables that were allocated for it above its Q register have their space deallocated. Next time the procedure is called, the local variables will be reallocated at new addresses, and will of course have new values. Everything below Qi -- including DB-Qi and DL-DB -- is not automatically changed by procedure calls and returns. The compilers allocate global variables between DB and Qi, so if you try to use that space, you'll interfere with them, but you can safely (fairly safely) put stuff between DL and DB, and unless you explicitly change it, it'll remain the same no matter how many procedures you call or exit. What IS the DL-DB area? It is just a part of the stack, containing memory locations belonging to your process (like DB-Qi or Qi-S). There are several rules, however, that you must follow to access this area: * The locations in this area have NEGATIVE addresses (since they're below DB and all stack addresses are DB-relative). This means that they can only be accessed directly using SPL, and you have to write special SPL interface procedures to access them from other languages. * By default, any program has less than 100 words of space available between DL and DB. If you want more (as apparently you do), you can run your program with ;DL=nnn to ensure that at least nnn words of DL-DB space will be available. Even better, you can use the DLSIZE intrinsic to dynamically expand and contract your DL-DB space: CALL INTRINSIC "DLSIZE" USING -1280, GIVING DLSIZE-GRANTED. I usually use DLSIZE instead of running my program with ;DL=, not just because it allows me to dynamically change the amount of space I have allocated, but also because it saves me embarrassment when I forget to :PREP my program with ;DL=. * Finally, although the system left the entire DL-DB area for "user applications", from MPE's point of view V/3000 and the like are also "user applications", and some of them are quite liberal with their DL-DB space usage. V/3000 is the big culprit, so I'd suggest that you avoid using the DL-DB area in programs that use V/3000. If you don't use V/3000, you should still stay clear of the area between about DL-16 and DL-1, which is used by SORT/MERGE, COBOL, and the FORTRAN formatter. So the point is that, once you've allocated enough space with ;DL= or DLSIZE, and have made sure that you're not colliding with anybody else (like V/3000), all you need to do is write a few small SPL procedures: $CONTROL NOSOURCE, SEGMENT=ADDR'WALDOES, SUBPROGRAM, USLINIT BEGIN PROCEDURE MOVEFROMADDR (BUFF, ADDR, LEN); VALUE ADDR; VALUE LEN; INTEGER ARRAY BUFF; INTEGER ADDR; INTEGER LEN; BEGIN INTEGER ARRAY RELATIVE'DB(*)=DB+0; MOVE BUFF:=RELATIVE'DB(ADDR),(LEN); END; PROCEDURE MOVETOADDR (ADDR, BUFF, LEN); VALUE ADDR; VALUE LEN; INTEGER ADDR; INTEGER ARRAY BUFF; INTEGER LEN; BEGIN INTEGER ARRAY RELATIVE'DB(*)=DB+0; MOVE RELATIVE'DB(ADDR):=BUFF,(LEN); END; END. Saying CALL "MOVETOADDR" USING -1023, MY-STUFF, 100. will move the first 100 words of MY-STUFF to the 100 locations starting with -1023; similarly, CALL "MOVEFROMADDR" USING MY-STUFF, -1023, 100. will move the 100 words starting with -1023 to MY-STUFF. In the above examples, "-1023" is the address of the DL-DB chunk you're using. It's a negative number like all DL-DB addresses, and it could either be a constant (if you always know that words -1023 through -924 are the ones that you'll use), or a variable. As you may have noticed, MOVETOADDR and MOVEFROMADDR move to and from any arbitrary address in the stack, with no concern as to whether it's between DL and DB or not. They are simply interfaces to SPL's arbitrary address handling capability; the COBOL program assigns the addresses, allocates the space, etc. Many such things that can "only be done in SPL" may easily be implemented in other languages by writing very simple SPL procedures that can be called from COBOL, FORTRAN, PASCAL, etc. programs. Q: How can I determine all the UDC files :SETCATALOGed on my system? Also, can I group UDC files together to improve performance? A: Starting with T-MIT (I believe -- it may have been T-delta or something like that), HP allows a user with SM capability to say :SHOWCATALOG ;USER=@.@ to show all the UDCs files currently set on the system level. That means all the files set using ":SETCATALOG ;SYSTEM", something that any user could always do without SM using the :SHOWCATALOG command. How incredibly useful. What you want -- and quite reasonably, I might add -- can NOT easily be done with any HP command. It should be, but it can't. There's no justice in the world. Fortunately, this information is not so hard for you to get at yourself. All the information on who has what UDCs set is kept in COMMAND.PUB.SYS, an ordinary file with a fairly simple format (described, in its usual incredibly readable style, by the System Tables Manual in Chapter 15). Simply put, * Record 0 is a header. * Any other record with a 1 in its word #1 is a "user record", which contains the user and account for which a UDC file (or files) is set. The user name starts in word 2, the account name in 6; account-level UDCs have an "@" in the user field, and system-level UDCs have an "@" in both the user and account. * Word #0 of any "user record" contains the record number of the first "file record" for this user. Each file record's word #0 contains the record number of the next file record belonging to this user, until the user's last file record, which has a 0 in its word #0. Each file record contains the filename starting with word 2. So, all you have to do is read through COMMAND.PUB.SYS, finding each user record and traversing its linked list of file records. In fact, that's exactly what the following quickie SPL program does: $CONTROL NOSOURCE, USLINIT << )C( COPYWRONG 1985 BY VESOFT. NO RIGHTS RESERVED. >> BEGIN INTRINSIC FOPEN, FREADDIR, PRINT; INTEGER ARRAY REC(0:19); INTEGER FNUM; BYTE ARRAY FILE(0:35):="COMMAND.PUB.SYS "; DOUBLE RECNUM; FNUM:=FOPEN (FILE, 1); RECNUM:=1D; << 0th record is just a header >> FREADDIR (FNUM, REC, 20, RECNUM); WHILE = DO BEGIN IF REC(1)=1 THEN << user record >> BEGIN PRINT (REC(2), -16, 0); WHILE REC(0)<>0 DO BEGIN FREADDIR (FNUM, REC, 20, DOUBLE(REC(0))); PRINT (REC(2), -36, 0); END; PRINT (REC, 0, 0); << blank line >> END; RECNUM:=RECNUM+1D; FREADDIR (FNUM, REC, 20, RECNUM); END; END. For brevity's sake, I've omitted much of the prettifying, so all you'll see is the name of each user who has UDCs set, followed by the names of all his files, followed by a blank line. Clean it up as you will, and put it in some revered spot on your system (note: this printout will show the lockwords for any UDC files which were :SETCATALOGed filename/lockword). It's likely to come in handy. About performance: Run-time performance (i.e. how long it takes MPE to recognize and parse your UDC invocation) is affected solely by the number of UDCs (not UDC files) a particular user has set; in any event, UDC lookup and expansion are usually very fast. Performance at logon time can be rather poor because MPE must open and read every UDC file set up on the user's behalf. The order of entries in COMMAND.PUB.SYS doesn't matter; what does matter is the number of UDC files (very important, since there's an FOPEN for each one and FOPENs ain't cheap), the combined size of the UDC files, and the blocking factor of each. First, try merging several UDC files into one; after that, increase the blocking factor of each UDC file to minimize the number of I/Os necessary to read it. Q: I don't like the fact that during a system backup, you can't run most programs. True enough, if you :ALLOCATE a program before the backup, you'll be able to run it, but why is that necessary? It seems that the :RUN fails because it somehow has to update the program file -- why should it do that? :RUN looks like it ought to be a read-only operation. A: True enough, for the most part, :RUN need only read the program file -- get various control information (capabilities, maxdata, etc.) and bring the code segments into memory. However, for some very interesting optimization reasons, a little bit of information needs to be written to the file, too. The principle of HP memory management is that you can have more code and data than will fit into memory. Just because you have 2 Megabytes of memory doesn't mean that all the stacks, code segments, etc. must fit into 2 Meg; if memory is full and you still need more, HP's Virtual Memory Manager will make room by getting rid of a segment that's already in memory but hasn't been used in a long time. Then, if the segment you've just taken out of main memory is needed again later, it'll be brought in when needed and some other segment will be thrown out. Note that I said that an old segment would be "removed from memory". You've probably heard this as a segment being "swapped out" -- if you're going to take, for instance, a process's stack out of main memory, you can't just throw it out; you have to copy out to disk, so that when it's needed again, it can be brought back into main memory with exactly the same contents as it had when it was taken out of main memory. Whenever the space currently used by a data segment must be used by something else, the data segment must be written out to disk for future reference. Now, say that the least recently used segment in memory is not a data segment, but rather a code segment. It is a fundamental principle of HP's architecture that code segments are CONSTANT and unchangeable. If we need to re-use the space currently used by a code segment, we can just THROW IT OUT without writing its contents out to disk, since its contents are already stored on disk in the program file itself! To summarize, every segment that isn't currently kept in main memory must be stored somewhere on disk. * Data segments -- which can change -- have a special place on disk allotted to them (this is what the "virtual memory" you configure is for), and any time they're removed from main memory, they have to be saved in this place on disk. * Code segments, being constant, are more efficient: when you need to get them out of main memory, you don't have to write them out to disk, since their contents are already stored on disk in the program file. What does all this have to do with the question at hand? Consider for a moment what a code segment must contain: * First, it must have the actual machine code that belongs to the segment. This machine code, of course, never changes until you recompile the program. * Furthermore, it must also contain links to all the various other segments that are called from within it. Say an SL procedure called XYZ is called from the segment -- the segment must contain the segment number of the segment that contains XYZ and the location of XYZ in that segment. This is information that can not be deduced at compile time or :PREP time, but must be filled in at :RUN time (since that's when all SL procedure references are bound). So, the problem that HP faced is that each segment had to contain information (in a table called the STT, Segment Transfer Table) that was not known until the program was :RUN, and even then could change from one :RUN of the program to another! Now, the only thing that makes HP's efficient code segment handling plan work is that the code segment in memory must be EXACTLY like the code segment in the program file on disk, since the segment may later have to be read in from the program file. Thus, if the STT of every segment has to be initialized every time the program is :RUN (which it does) and if the STT is kept in the segment (which it is), the STT must be initialized in the program file as well in the in-memory code segment. So there's the explanation. Every time a program is loaded, the STTs of all the segments (and also some other things) must be initialized, and so the program file has to be updated. That's why the loader opens the program with read/write access, and when the file is being :STOREd, this kind of thing is forbidden. On the other hand, if the program is already being :RUN or has been :ALLOCATEd, then it's already been loaded, and the loader needn't re-update the STTs, and thus needn't open the file for read/write. Incidentally, have you ever wondered why program files must only have one extent? HP demands that the disk images of all code and data segments must be contiguous, so they could be read in one I/O. The only way to make sure that a code segment in a program file is contiguous is to make sure that the entire program file is contiguous, which means that it must have one extent. Q: What's the difference between $STDIN and $STDINX? A: Good question. An equally good question, incidentally, is: "WHY the difference between $STDIN and $STDINX?", but I'll get to that in a moment. One of the key points of the HP file system is that all sorts of files are accessed in pretty much the same way. There's no special intrinsic for writing to the line printer -- you just open a file on device class "LP"; similarly, the terminal is accessed using file system intrinsics, too, using the special files "$STDIN", "$STDINX", and "$STDLIST". Now, all disc, tape, etc. files have "end of file" conditions. A typical file read loop in your program might be: 10: READ FILE F IF END OF FILE ON F GOTO 20 process the record that was read from F GOTO 10 20: ... You read records from the file until you hit an end of file condition. Now, let's say that using a :FILE equations you've redirected file F to $STDIN -- instead of getting the data from disk, you want your program to read it from the terminal. You have to have some way of indicating an end of file condition that would look just like the end of a disk file. Now, the actual difference between $STDIN and $STDINX is only that: * On $STDIN, any record that starts with a ":" is an end of file. * On $STDINX, only records that start with ":EOD", ":EOF: ", or (in batch jobs) ":JOB", ":EOJ", or ":DATA" constitute an end of file. Based on this alone, it's pretty obvious that all on-line programs should read from $STDINX (which, of course, is NOT the default for FORTRAN, COBOL, etc.). After all, the last thing that you want to do is abort with a tombstone if a user accidentally enters a ":". In fact, I dare say that in a session environment, $STDIN is quite thoroughly useless, and $STDINX should have been the default and perhaps the only option available. If this is so, though, why does $STDIN even exist? Well, remember that in 1972, when the 3000 was first built, batch use was far more prominent than on-line use. A typical job stream then (as now) might look like this: !JOB JOE.SCHMOE !RUN FOOBAR 123 << all this stuff is being read from $STDIN >> 456 789 !RUN FROBOZZ XYZ123 << another program, also reading $STDIN >> !EOJ The Powers That Be decided that it would be nice if the "!RUN FROBOZZ" (which the :STREAM command translates into ":RUN FROBOZZ") terminated the previous program (if it was still running) as well as starting the next one. So, any command that started with a ":" would cause an EOF on $STDIN (if the previous program was still reading from $STDIN), and then would be executed as an ordinary MPE command. Similarly, if a :JOB, :EOJ, or :DATA was encountered in the input stream, an EOF would be triggered even on $STDINX. That way, if several job streams are submitted at once (in 1972, this was quite common when jobs were fed in using punch cards), it would never happen that one job's input requests would receive as input the commands of another job. So both the ":" termination of $STDIN and the :EOD/:EOF:/:JOB/:EOJ/ :DATA termination of $STDINX were done to make sure that MPE commands would not be inadvertently fed as input to a program that is expecting normal data. The distinction between $STDIN and $STDINX was invented because although it was felt that the ":" end-of-file condition would be more useful (which today is definitely not the case), some programs (e.g. EDITOR) needed to be able to accept lines starting with ":". So, I've told you the WHAT and the WHY. Now, all that's left is a bit of WARNING: * First of all, an end-of-file on $STDIN isn't just a temporary thing. If a program gets an EOF while reading $STDIN, all subsequent read requests by that program against $STDIN will also get an EOF. In other words, once a program gets an EOF on $STDIN, it won't be able to do any more $STDIN input; also, any program that gets an EOF on $STDINX won't be able to do any more $STDIN or $STDINX input. * Furthermore, if a program gets a $STDIN[X] EOF, ALL OTHER PROGRAMS IN THE PROCESS TREE WILL ALSO GET AN EOF. In other words, if you run SPOOK (which reads $STDIN) from within QEDIT or TDP (which read $STDINX) and enter a ":", SPOOK will terminate with an EOF. Whenever you run SPOOK again from the same execution of QEDIT/TDP, it will immediately get an EOF. To reset the EOF condition, you must get back to MPE and then re-run your program; OR, you may do a ":STREAM" command and immediately enter a ":" in response to the prompt. It turns out that :STREAM magically resets the EOF condition on $STDIN. * If you enter a ":EOF:" (followed by a space or carriage return), you get a "super-EOF" condition. Getting out to MPE will NOT fix it -- an end-of-file is caused on $STDIN and $STDINX of all processes, including MPE itself. As soon as you get out to MPE, you get automatically logged off; even hitting [BREAK] to get out to MPE will automatically abort the broken program and sign you off. * Finally, let me expose a couple of errors that have been floating around HP documentation: - In session mode, only ":EOD" and ":EOF:" cause an end-of-file on $STDINX. :JOB/:EOJ/:DATA/:BYE/:HELLO/etc. do NOT. - In job mode, ":EOD", ":EOF:", ":JOB", ":EOJ", and ":DATA" cause an end-of-file on $STDINX. :HELLO does NOT. - Some manuals say that if you read less than 4 characters from $STDINX, anything that starts with a ":" cause an EOF (since you can't enter an :EOD if less than 4 characters are being read). This is WRONG -- there's no way to enter an EOF if you're reading less than 4 characters from $STDINX. So, there it is, in all its dubious glory. If you've got a choice, always use $STDINX (at least in session mode). Q: Many of my programs need a lot of stack space for things like an internal sort or for V/3000 screen handling. What kind of performance penalty would I suffer if I ran them with a high maxdata -- 20,000 or 25,000 -- just to be safe? A: Your question brings up a very interesting point -- what does the ;MAXDATA= parameter do, after all? All the variables and arrays that your program uses are put in your "stack data segment". This is just a segment that's allocated on your behalf by the system; it can be up to 32640 (32768-128) words, of which some amount is always used by the system for things like open file information. Initially, the stack is made with room for all your global variables (which are allocated throughout the duration of the program) and some extra space for a small amount of local variables. As local variables are allocated (by a call to any procedure, of which SORT and V/3000 are only particularly conspicuous examples), the system checks to see if there's enough room for them. If not, it expands the stack to make room. This is where ;MAXDATA= comes in. If the expanded stack would be bigger than the size given as the MAXDATA, the program aborts with a stack overflow. There are two reasons for having a MAXDATA in the first place. One is that MAXDATA specifies the space allocated for the stack data segment in VIRTUAL MEMORY (not in main memory, but out on disk where the segment will go when it's swapped out). For all the tens of thousands of users out there who're running out of virtual memory, this is a good reason to keep MAXDATAs low. The other reason is that if you care about saving REAL MEMORY, specifying a low MAXDATA= can prohibit your stack from getting too big; when the stack overflow happens, you'll know that your program is a memory hog and maybe you'll be able to do something about it. What setting a low MAXDATA doesn't do is actually save you any real memory. If your program needs only 5,000 words, it'll use only 5,000 if its MAXDATA is 10,000 or 30,000; if your program needs 25,000 words, setting the MAXDATA lower than that will only make it abort. So, my advice is: If you get a stack overflow, push the MAXDATA way up -- to 20,000 or 30,000. If you really want to save memory, you have to manually optimize your program to minimize its memory usage. So, there's the answer to your immediate question; but, like many answers, it actually raises more questions than it solves. First of all: What's the easiest way of increasing a program file's MAXDATA? Well, you can re-:PREP it if you want to, but that requires the USL file to still be present and also takes quite a bit of time. Fortunately, a program's MAXDATA value is just one word in the program file's 0th record; various programs are available to change it, including some Contributed Library utilities (like MAXCAP) and VESOFT's MPEX/3000 %ALTFILE ;MAXDATA= command. If demanding more stack space dynamically increases stack size, does relinquishing space (say, by exiting a procedure that's allocated a lot of local variables) decrease stack size? No. If you want to actually decrease stack size, you'll have to call the ZSIZE system intrinsic, passing to it a stack size of 0 (CALL INTRINSIC "ZSIZE" USING 0). This'll cut down the actual memory usage of the program to only the amount of space that the program really needs. Watch out, though -- this operation may take more processor time than will be saved by the decreased memory use! OK, now that I've explained exactly what MAXDATA= does, what does the STACK= parameter do? Well, as I said, MPE initially allocates your stack data segment to be large enough to fit your global variables, but not much more. As local variables are allocated and the allocated stack size is exceeded, the stack will be dynamically expanded. Theoretically, a dynamic stack expansion can be a fairly inefficient thing (it might actually require a swap-out and swap-in of your stack). So, if you know that your program will use at least 10,000 words of stack space for most of its life, you can tell MPE to allocate that much space initially by specifying ;STACK=10000 on the :RUN or :PREP command. Practically, though, I don't think this is worth your time. Oh, yes, one minor confession: I lied a bit when I talked about V/3000 and SORT allocating local variables on the stack. SORT actually does stick all its stuff on top of the stack, above the Q register, but V/3000 puts it in the so-called "DB negative area", between DL and DB. This fact is actually quite irrelevant to our discussion; I only mention it so that those who know it won't think I'm trying to pull a fast one. Q: I've got a program that I run a lot in batch mode. It asks for several lines of input data, and then chugs along to produce a report. Sometimes, it'll find that some of the data is invalid, and then it terminates (without calling QUIT or anything -- just TERMINATE) immediately after seeing the incorrect piece of data. Unfortunately, MPE insists on reading the next piece of data (which is, of course, NOT a valid MPE command) as input for the command interpreter; MPE sees that it's an invalid command, and promptly aborts the job stream, which isn't what I want. In other words, if my job stream looks like !JOB FROBOZZ,USER.PROD !RUN MYPROG 01/01/86 A037 F999 127.44 !RUN MORESTUF !EOJ and MYPROG finds that the "A037" is an invalid input, MYPROG will then terminate, and MPE will try to execute "F999" as an MPE command. What can I do? I tried putting a "!CONTINUE" in front of the "!RUN", but it doesn't help. A: Your desire -- to have MPE throw out all the remaining MYPROG input until it sees a valid MPE command, like "!RUN MORESTUF" -- is perfectly legitimate. In fact, it's so legitimate that MPE actually does exactly what you want! But not all the time. The key distinction, amazing as it may seem, is whether the program being run HAS EVER FOPEN'ED $STDIN. In other words, has it ever called the FOPEN intrinsic with the appropriate filename or foptions that indicate either $STDIN or $STDINX? If it has, then when the program terminates, MPE will read lines from the job stream and throw them away until it finds one that starts with a ":" (the :STREAM facility translates the "!"s in your stream file into ":"s); then, it will start executing MPE commands from that point onward. On the other hand, if your program NEVER FOPENs $STDIN or $STDINX -- which means that it reads input using the READ or READX intrinsics -- then MPE will NOT throw away any invalid input, but will start trying to execute MPE commands starting with the first input line that wasn't read by the program (in your case, the "F999"). A good test of this is to run the following job stream: !JOB TESTER,USER.PROD;OUTCLASS=,1 !CONTINUE !FCOPY FROM=$STDIN;TO;SUBSET=0,1 TESTING ONE TWO THREE !SHOWTIME !CONTINUE !FCOPY TESTING ONE TWO THREE !SHOWTIME !EOJ The first run of FCOPY will read one record (that's what the ";SUBSET=0,1" is there for) from $STDIN and print it to $STDLIST. Then, FCOPY will terminate, since it was only asked to read one record; but, since the "FROM=$STDIN" made FCOPY do a FOPEN of $STDIN, the ONE, TWO, and THREE will be thrown away, and the job stream will continue executing starting with the :SHOWTIME. The second run of FCOPY reads TESTING as an FCOPY command; FCOPY sees that it's an invalid command, and will terminate (just like it does in the first case after reading one line). However, since in this run FCOPY has never FOPENed $STDIN (it usually reads its input by calling the READX intrinsic, which doesn't require an explicit $STDIN FOPEN), the "ONE", "TWO", and "THREE" won't be thrown away. MPE will read the "ONE", see that it's not a valid MPE command, and will abort the job stream. The "!CONTINUE" won't help, since it only affects the next command, which is "!FCOPY" -- since there is no "!CONTINUE" before the "ONE", an error reading the "ONE" will abort the stream. One thing you have to realize, by the way, is that some languages automatically FOPEN $STDIN or $STDINX on your behalf. FORTRAN, PASCAL, and BASIC, for instance, do -- I'm not sure about COBOL or RPG. SPL, on the other hand, doesn't, so if your program is written in SPL and you want to take advantage of this "throw away unread input" feature, you have to make sure that your program does the FOPEN itself. All it has to do is have a line like FOPEN (,%40); << foptions %40 means $STDIN >> at the beginning -- it can just throw away the result of the FOPEN and keep calling READ or READX. As long as the FOPEN is done, you'll have what you want. Q: Can you tell me what the following file codes are for: VREF, RJEPN, PCELL, PCCMP, RASTR, TEPES, TEPEL, SAMPL, MPEDL, TSR, TSD, DSTOR, TCODE, RCODE, ICODE, MDIST, MTEXT, VCSF, TTYPE, TVFC, NCONF, NTRAC, NLOG, and MIDAS. Perhaps some of them are created by products we don't have. A: Often, when they have nothing better to do, HP engineers invent new filecodes to intellectually stimulate and confuse the user community. This phenomenon, known technically to psychiatrists as "cybernetic coding release", is thought to be a beneficial thing, something that safely channels frustrations that would otherwise go towards things like putting unneeded SUDDENDEATHs into MPE code or purging random files from your system. I called up my friend Wilfred Harrison at HP -- an unimpeachable source who would never lead me astray -- and he told me the following: * VREF stands for View REFormat specification files, made by HP's REFSPEC utility. * RJEPN are RJE PuNch files, created by Remote Job Entry/3000. * ICODE, RCODE, and TCODE files are created by HP's RAPID/3000 system (INFORM, REPORT, and TRANSACT respectively), which may or may not be very fast on your system. * PCELL stands for Padded CELL, and denotes a file which contains various programs that have gone crazy and had to be locked up for their own good. VESOFT's MPEX/3000 is indispensable for handling these files, since with it you can say: %LISTF @.@.@(CODE="PCELL") and find all those screwed-up files before they break loose and do something nasty. * SAMPL files are SAMPLer/3000 (now known as APS/3000) log files. * MIDAS files turn to gold any computer system on which they're present. Just try: :BUILD POOF;CODE=MIDAS These are, unfortunately, the only ones I managed to find out about. As I hear of more, I'll be sure to let you know. Q: Once upon a time, I was using EDITOR and wanted to look at one of my files. I said /TEXT FFF and EDITOR replied *23*FAILURE TO OPEN TEXT FILE (0) END OF FILE (FSERR 0) Naturally, I was rather taken aback; FFF was supposed to be a program of a thousand-odd lines. I entered :LISTF FFF,2 and up comes ACCOUNT= VESOFT GROUP= DEV FILENAME CODE ------------LOGICAL RECORD----------- ----SPACE---- SIZE TYP EOF LIMIT R/B SECTORS #X MX FFF 80B FA 1076 1076 3 360 8 8 Most distressing indeed! Now, to add to my confusion, the operator at this point warmstarted the system, and when it came back up, my file was back! I /TEXTed it in using EDITOR, and there it was, all 1076 lines of it. Naturally, I'm glad to have my file back, but tell me: what's going on here? A: As best I can tell, you have fallen victim to the infamous INVISIBLE TEMPORARY FILE. Actually, it's not quite invisible, but the way HP designed the system, it's pretty hard to see if you're not looking for it. When you build a file, you have the choice of two DOMAINS in which to build it. You can create it as a PERMANENT file, which means it stays around forever (or at least until you purge it); most files, including your FFF program, are like that. You can also build a file as a TEMPORARY file, which means that the file is automatically deleted when you log off; also, different sessions can have temporary files with the same names without any problem (since each session can only access its own temporary files). Say your program needs to build a file that it wants to :STREAM; if it built a permanent file called, say, STRMFL.PUB.DEV, then it might conflict with the same file built at the same time by another user who's running this program. If it builds STRMFL as a temporary file, however, no such problem will arise. As an additional bonus, if the program doesn't delete the file itself, it'll automatically vanish (and have its space freed for future use) when the session logs off. So, both permanent and temporary files have their place, although most of the files you deal with are permanent files (after all, the computer's job is to store data more or less permanently). The problem that arises is: when you open a file called XYZ, does this mean the PERMANENT file XYZ or the TEMPORARY file XYZ? There may, after all, be both a permanent and a temporary file with the same name. Therein lies the root of your problem. MPE gives you three ways to open an already-existing file: * You can demand that MPE open the PERMANENT file with a given filename. This corresponds to the :FILE command option ;OLD, and to an FOPEN call with bits .(14:2) of FOPTIONS set to 1. * You can demand that MPE open the TEMPORARY file with a given filename. This corresponds to :FILE ;OLDTEMP and FOPTIONS.(14:2) = 2. * Or, you can tell MPE to try to open the TEMPORARY file with this name, or, if no such file exists, to open the PERMANENT file with this name. Now, most system programs, including EDITOR, FCOPY, and MPE's :STREAM command, use the third option -- try to open the temporary file first, and the try the permanent file. It's convenient for them to do it, since then the same /TEXT command or :STREAM command can work on both temporary and permanent files -- they don't need to have special keywords like /TEXT FFF,TEMP or :STREAM FFF,TEMP. Unfortunately, it can also be rather confusing, since the :LISTF command only prints permanent files and the :PURGE command by default purges only permanent files. So, that's my diagnosis of your troubles. You must have somehow gotten a TEMPORARY file called FFF, which didn't have any records in it. When you did a /TEXT FFF in EDITOR, EDITOR saw this temporary file, and gave you an error because it -- the temporary file -- was empty. However, :LISTF showed you the permanent file, which wasn't empty at all. Fortunately for you, the system went down and all your session's temporary files perished. When you signed back on, the temporary file wasn't there and the /TEXT once again accessed the permanent file. So, the lesson: be wary of the INVISIBLE TEMPORARY FILE. If any eerie things like this happen to you, run LISTEQ5 or do a :LISTFTEMP to see whether there are any temporary files "shadowing" your permanent files. (While you're at it, you might also check :LISTEQ to see whether there are any improper file equations for the file you're trying to open. But that is another story.) And, above all, remember: things are not always as they seem. Q: I've heard that HP's new Spectrum machine isn't a stack architecture machine like the 3000. If this is so, what's going to happen to all the variables and stuff that I keep in my stack? Will I still be able to write recursive procedures, which require a stack to operate? What will they use instead of a stack. A: Unfortunately, labels like "stack architecture", "fourth- generation language", "relational database", and such often obscure far more than they clarify. When HP's says that their new Spectrum (or Series 950, or Precision Architecture, or whatever they're calling it this week) is not a stack architecture machine, they mean something quite different from what one might guess. A stack is a data structure onto which you can push objects and then pop them in reverse order. In other words, it's not like a variable, storing into which overwrites the old value, or like an IMAGE master dataset, from which data can be retrieved by its key rather than by its order of insertion. A stack is useful for situations in which you want to do something, and, when you're done with it, return to the thing you were doing immediately before that. What are good applications for a stack? Well, one that was recognized very early was for saving RETURN STATE INFORMATION. Say that instruction 175 in subroutine A decides to call subroutine B. When subroutine B is done, you want to return back to instruction 176; but, how is subroutine B to know that's where you want to go? Early machines would save the return instruction address in a special register, and the "RETURN FROM SUBROUTINE" instruction would branch to the address stored in that register. But, there was only one register -- when subroutine B called subroutine C, the return address for the A-to-B call would be forgotten, and the return from subroutine B would fail. Other computers, like the HP 2100, were wiser -- they stored the return address of a subroutine IN A MEMORY LOCATION AT THE FRONT OF SUBROUTINE BEING RETURNED FROM. In other words, if instruction 175 in subroutine A would call subroutine B, whose first instruction is 305, the return address (176) would be saved at location 304. Then, when subroutine B called subroutine C, the C-to-B return address would be saved at the header of subroutine C, thus not overwriting the B-to-A return address! A return-from-subroutine instruction would just branch to the address stored in the header of the routine that's doing the return. However, the most general and the most convenient method for keeping track of return information has been found to be a STACK. Every subroutine call would push the return address (as well as other information, like the processor status word, various registers, etc.) onto the stack; every subroutine exit would pop an address from the stack and branch to the popped address. Clean, simple, powerful -- this way even RECURSIVE routines, in which the same routine can call itself many times in a loop, would work, as long as you had room on your stack. Another, related use for stack structures has historically been the keeping of subroutine parameters and subroutine-local variables. Note the similarity with return information -- as a new routine is called, space is allocated on the stack for its local data; when it exits, the local information should be deallocated. Furthermore, since subroutines are always exited in reverse of the order in which they were entered, the stack is a perfect structure for keeping this information. Finally, arithmetic expressions (like A*B+C*D) have often been evaluated using a stack. A would be pushed onto the stacked, then B, then the top two stack elements would be multiplied, yielding A*B; then, C and D would be pushed, the top two stack elements would again be multiplied (= C*D) and then the new top two stack elements (A*B and C*D) would be added to yield the procedure result. Just like on your HP calculator, expressions on the HP3000 are evaluated by pushing operands onto the stack and then using the operators to pop them and leave the results in their place. These three things are the primary uses of the data structure called a "stack" on the HP3000 and other machines. However, the overwhelming majority of both STACK-ARCHITECTURE and the opposite, REGISTER-ARCHITECTURE machines, use stacks for the first two purposes (subroutine return information and subroutine parameters and local variables). Stacks are perfectly tailored for these purposes, and everything from your PC to a VAX to HP's new architecture uses them. You'll still be able to do all the recursion you want, because return addresses will always be kept in a stack, as is needed for recursion; or, conversely, HP was forced to keep all the returned information in a stack architecture, since without it useful mechanisms such as recursion would be unavailable. The distinction between STACK- and REGISTER-ARCHITECTURE machines comes mainly in the third area: where the temporary data used in evaluating expressions is put. In the HP3000 it's pushed on the stack by some instructions (e.g. LOAD) and popped by others (e.g. ADD and STOR). In register machines (like most modern machines, such as the 8080, 68000, and the HP3000/950), the temporary data is put in special high-speed machine registers. In other words, instead of pushing A, B, C, and D on the stack in order to evaluate A*B+C*D, the 950 might load A into register 0, B into register 1, C into register 2, D into register 3, multiply registers 0 and 1 (leaving the result in 0), multiply registers 2 and 3 (leaving the result in 2), and then add registers 0 and 2 to yield the result of the expression. This is actually a VERY GOOD THING, since registers are usually much faster to access than a stack, and a large number of registers can substantially cut down on costly memory accesses. HP may or may not do some other things to use its registers better. For instance, it might put some procedure parameters or frequently used local variables into registers, again striving to minimize memory (non-register) accesses. Of course, on all computers there's a limited number of registers (usually from 8 to 32), so anything that won't fit in registers will be kept in a memory stack (including perhaps even the intermediate values of complicated expressions). The important thing, though, is that it's all done behind your back -- the REGISTER/STACK ARCHITECTURE distinction here is only relevant for performance considerations. So that's what'll happen to stack data structures in HP's new machines, and that's why you'll still have recursion, dynamic allocation of subroutine-local storage, and all the other things that are so good about stack architectures. What about the other stuff HP keeps in the stack, like global variables? Well, there you're the victim of an unfortunate semantic muddle in the HP3000. What HP calls a "stack", many other computers call your "data segment" or "data space". Global variables aren't really kept in a "stack", in that they're never pushed or popped. Rather, they're stored in a data segment that happens to also contain your return address/local variable stack. In the 950, you'll still have your main data segment (how could you not?); in fact, it might still have the return/local stack stored inside it. HP probably won't call it a stack, but that's only a name; the important thing is that it will support all the operations you've grown to know and love, and probably (cross your fingers) do them a lot faster, to boot. So, just as the moral of the last story was "don't believe everything you read", the moral of this one is "things are seldom as they seem." Or, perhaps, "put not your trust in buzzwords." Q: I hear that HP's new 3000 Series 950s aren't going to support SPL. What about all my programs and vendors' programs that are written in SPL? Will we have to re-write them all? What if we've lost the sources? A: HP has built a well-deserved reputation for compatibility, both on the source-code and object-code levels. You can write a program, in SPL or in any other language and have it run on anything from a Series 30 to a Series 70 without any recompilation. The 950 has a radically different instruction set (RISC = Radically different Instruction Set Computer...), so you wouldn't be able to feed your 3000 program files, written in any language, to the CPU, just like you couldn't feed VAX or IBM programs to the 950's CPU. However, in addition to providing FORTRAN, PASCAL, COBOL, BASIC, and RPG compilers that will generate code that the 950 CPU can handle, HP also allows you to run 3000 code that will be EMULATED by the 950. In other words, any code (except for some privileged operations) that the 3000 can execute DIRECTLY, the 950 can execute in EMULATION MODE, essentially a very fast (but sometimes not fast enough) interpreter. They don't call it a Really Intelligent Super-Computer for nothing. Consider, for a moment, the SPL compiler. Forget the aura of glory and mysticism that comes of our calling it a "compiler". It's just a program, a program that takes as input one data file (which happens to contain SPL source code) and generates as output another data file (which, it turns out, contains 3000 machine code in USL format). In fact, SPL doesn't even use privileged mode; it's just another HP3000 program, written in the HP3000 instruction set, that reads an input file and writes an output file. The point, you see, is just like you can run your 3000 application system on the 950 without recompiling it, so you can run the SPL compiler on the 950. And, the USL file that SPL.PUB.SYS outputs -- on the 3000 or the 950 -- will contain 3000 code, which can also be run on the 950. The only thing you have to realize is that both SPL.PUB.SYS and the program that it outputs will have to be run in Compatibility Mode; however, they can be run just as easily as any Compatibility Mode program written in COBOL, PASCAL, FORTRAN, or what have you. After all, it would be a Really Idiotic and Silly Computer that would single out only SPL-generated code for unfavorable treatment while allowing the same instructions generated by COBOL, PASCAL, or FORTRAN compilers to run just fine. Of course, since HP's SPL compiler hasn't been re-written to generate Native Mode code, its output can only be run in the relatively slower Compatibility Mode. If you want your SPL programs to run in Native Mode, you'll have to use SPLash!, the new third-party compiler that the people at Software Research Northwest (super-programmers like Steve Cooper, Jason Goertz, Wayne Holt, Stan Sieler, and Jacques Van Damme) are putting out -- write to them at: PO Box 16348 Seattle WA 98116 USA. If you BOTH want it to run in Native Mode AND don't want to use SPLash!, then you'll have to re-write your programs in some other language, preferably C or PASCAL (if you really want to have fun, write them in RPG). But, remember that this is essentially only a performance improvement; unless you use PM, your SPL programs will run intact in Compatibility Mode on the 950, with or without recompilation. I'd guess that for starters, most HP users, vendors, and HP itself will keep much of their SPL code in SPL and run it in Compatibility Mode, if need be fixing some of the spots that use PM. Then, over several years (depending on how important performance is and how good a job HP does in making its Emulation Mode fast), most of the code will migrate into Native Mode. But, I predict that ten years from now there'll still be code on Series 950's that runs in SPL, and nobody will be at all the worse for it. And that's a Rather Insightful Soothsayer's Cerebration, if I may say so myself. Q: I wanted to redirect the output of my program to a disc file, so I said: :RUN DAD;STDLIST=LFILE,NEW However, DAD creates several son processes, all of whose output I want to send to LFILE as well. What happened was that only one of the processes' output was put in LFILE, and the rest went into the bit bucket. What's up? A: What you've run into here is an artifact of the way the 3000 takes care of the ;STDLIST= parameter -- a restriction that makes the ";STDLIST=xxx,NEW" feature far less useful that one might think. Let's say that you have three processes -- DAD, KID1, and KID2. DAD is run with $STDLIST redirected to some file, and it then creates KID1 and KID2. When MPE opens DAD's $STDLIST file (MPE always opens a $STDIN file and a $STDLIST file on behalf of each process), it will open the file LFILE as a NEW file. Remember what happens when you open a file as NEW -- the system won't look for the file in the temporary or permanent file directory, and in fact won't even put the file in the directory until you close it. If you FCLOSE it mode 1, it'll save it as a permanent file; if you FCLOSE it mode 2 (as ;STDLIST=xxx,NEW does), it'll save it as a temporary file; if you FCLOSE it mode 0 or 4, it won't save it at all; but IN ANY CASE, WHEN YOU OPEN A FILE AS NEW, IT WON'T BE PUT IN THE DIRECTORY UNTIL YOU FCLOSE IT. So DAD's now running, with its $STDLIST redirected to the file LFILE -- a file that has a file label, has disc space, but DOESN'T have a directory entry (either permanent or temporary). Now, KID1 is created. MPE's smart enough to realize that KID1 should inherit DAD's $STDLIST, so when it creates KID1, it's essentially as if it was also run with ";STDLIST=LFILE,NEW". However, IT'S NOT THE SAME 'LFILE'! Since KID1 is also told to open LFILE as new, it'll create its own new file, also called LFILE, which won't cause any sort of conflict since DAD's LFILE hasn't been put into a directory yet. In fact, even if KID1 WANTED to open DAD's LFILE, it couldn't, since DAD's LFILE isn't in any directory. The same happens to KID2. It also opens a new file called LFILE -- instead of what you wanted, which is a single LFILE with all three processes' outputs, you get three LFILEes, one for each process. You see, the system took you at your word when you said that you wanted the $STDLIST to be a NEW file called LFILE, and opened each LFILE as a NEW file. So, as long as DAD, KID1, and KID2 are running, they're writing all their output to their own $STDLIST files. Now, say KID1 dies; at THIS point, its LFILE file is closed, and put into the temporary file directory. When KID2 and DAD die, and THEIR list files are closed, the system will also try to save them as temporary files, but since a temporary file called LFILE (from KID1) already exists, the saves will fail. Thus, when the entire process structure is done, the file LFILE will contain only KID1's output. So, that's what's causing your problem. The solution? Well, recall that the root of all your woes is that the system tried to open all the $STDLISTs as NEW files, which means that there was no chance for KID1 or KID2 to find DAD's $STDLIST. All you need to do is build LFILE as a temporary (or permanent) file BEFORE running DAD, and NOT specify the ",NEW" in the ;STDLIST= keyword. In other words, simply say :BUILD LFILE;REC=-250,,V,ASCII;CCTL :RUN DAD;STDLIST=LFILE to make LFILE a permanent file, or :BUILD LFILE;TEMP;REC=-250,,V,ASCII;CCTL :FILE LFILE,OLDTEMP :RUN DAD;STDLIST=*LFILE :RESET LFILE to make it a temporary file. Incidentally, this problem is NOT CONFINED TO PROGRAMS THAT DO PROCESS HANDLING! Exactly the same thing will happen if your program explicitly FOPENs $STDLIST -- if you then do I/O both to your normal $STDLIST (using, say, the PRINT intrinsic, or by calling the COMMAND intrinsic with a command like :SHOWTIME or :LISTF that outputs stuff to $STDLIST) and the FOPENed file, then the output to one of these files will get lost! This can be very bad for programs written in languages like FORTRAN, which always do their output using FWRITEs to an explicitly FOPENed $STDLIST file, rather than using the PRINT intrinsic to output to the system-opened $STDLIST. In conclusion, the ";STDLIST=xxx,NEW" option on the :RUN command isn't as good as it might seem. It'll only work right for you if the process does all its output to the system-opened $STDLIST, and doesn't run any sons or explicitly open $STDLIST itself. For safety's sake, I recommend that you always build the output file first and use ";STDLIST=xxx" without the ",NEW" option. Q: What are primary and secondary DB? I've heard a lot about them, and my SPL compiles all show "PRIMARY DB=%xxx; SECONDARY DB=%yyy" at the end, but I'm not sure what they are. I thought every process had only one DB register. A: The HP3000 is a 16-bit machine, with all (well, almost all) of its instructions encoded in 16 bits. For instance, a "LOAD DB+123" instruction -- which pushes the value at DB+123 onto the stack -- is represented as the 16-bit word %041173. Now, these 16 bits must contain a lot of information. They must indicate both the OPERATION to be performed (LOAD), and the PARAMETER on which it is to be executed (DB+123). What's more, since you might also say "LOAD Q+7" or "LOAD S-10", the PARAMETER must include the ADDRESSING MODE (e.g. DB+, Q+, or S-) and the ADDRESS VALUE (123). It is quite a challenge to compress all this information into a single 16-bit instruction. Consider the way the LOAD instruction is laid out (you can find this in your MPE V software pocket guide): bits 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 OPCODE(04) X I P ---------address--------- * The first four bits (04 octal, 0100 binary) indicate this to be a LOAD instruction; * Bit 4 indicates whether or not this LOAD uses an index register (used for array indexing); * Bit 5 indicates whether the LOAD should load the value at the given address or the value POINTED to by the value address (more about this later); * Bit 6 indicates whether it should load data from your data segment or your code segment; * And bits 7 through 15 contain further information about what address you want to load from. The address that you can load from is restricted to 9 bits! What's more, we still haven't determined whether this is a LOAD DB+ instruction (used to load global variables), a LOAD Q- (load a procedure parameter), LOAD Q+ (load a procedure local variable), or a LOAD S- (load an expression temporary value). All this must also be encoded in these 9 bits, while still leaving room for indicating exactly what address is to be loaded from! Thus, the HP3000 is not just restricted to a 16-bit address space; even the 64K bytes that you have in your stack can't be directly accessed because no instruction can access the entire address space. Back to the LOAD command, the full format of those 9 bits that contain the address is: * If P=0 (we're loading VARIABLES from our STACK), and bit 7=0, this is a LOAD DB+ instruction. The DB-relative address is given by bits 8 through 15, so we can directly load any value stored in DB+0 through DB+255. * If P=0, bit 7=1, and bit 8=0, this is a LOAD Q+ instruction. The Q-relative address is stored in bits 9 through 15, so we can load Q+0 through Q+127. * If P=0, bit 7=1, bit 8=1, and bit 9=0, this is a LOAD Q- instruction, with the address in bits 10 through 15, allowing us to get at Q-0 through Q-63. * If P=0, bit 7=1, bit 8=1, and bit 9=1, this is a LOAD S- instruction, the address is in bits 10 through 15, and we can access S-0 through S-63. * If P=1 (we're loading CONSTANTS from our CODE SEGMENT), then this is a LOAD P+ if bit 7=0 and a LOAD P- if bit 7=1. THIS is the key to the distinction between primary DB and secondary DB (as well as many other confusions and conundrums). All program global variables are allocated (at least in SPL) with DB-relative addresses (since DB always stays constant, unlike Q and S, which move around). If we allocated all our global variables contiguously from DB+0 up, we'd quickly run out of our 256 directly accessible words. What the compilers do instead is they allocate all SIMPLE VARIABLES (non-arrays) contiguously in the DB+ area; but, for every array, they allocate a 1-WORD POINTER CELL, which is initialized to point to the actual array data. Any access to an array will then go INDIRECTLY through the pointer cell, rather than specifying the array address directly in the LOAD (or STOR, etc.) instruction. So, if we have a program like $CONTROL NOSOURCE, USLINIT BEGIN INTEGER I; DOUBLE J; INTEGER K; INTEGER ARRAY L(0:9999); INTEGER M; ... END. then I will be put at DB+0 J at DB+1 and DB+2 (since it's still a simple variable) K at DB+3 a POINTER to L at DB+4 M at DB+5 When the program is run, the pointer to L will be initialized to the actual address of L (which now no longer has to be between DB+0 and DB+255, since it's accessed with a full 16-bit pointer), and code like "K:=L;" (where saying L without an index means getting the 0th element of L) will compile into LOAD DB+4,I << load the value POINTED to by DB+4 >> STOR DB+5 << store DIRECTLY into DB+5 >> Of course, fetching the 0th element of L will be slower than fetching, say, K, or M, because two memory addresses will be needed -- one to DB+4 and one to the location pointed to by DB+4. However, the quick, direct access mode would only allow us to access 256 words, not quite enough in today's world. To summarize, only locations from DB+0 through DB+255 can be accessed directly using a LOAD instruction, so compilers are careful to allocate these locations to simple variables and to array pointers. The area used by these variables and pointers is thus called PRIMARY DB. Array pointers point to array data, which can be anywhere in the stack, at any address, above or below 255. All this array data is called SECONDARY DB. PRIMARY DB can easily overflow if you have more than 256 words' worth of simple global variables and global array pointers; SECONDARY DB, on the other hand, can go all the way up to your maximum 30,000-odd words. There is only one DB register -- primary DB is accessed by DIRECT DB-relative addressing, whereas secondary DB is accessed by INDIRECT addressing, using primary DB values as pointers to the secondary DB data. Q: I'm streaming a job stream which has a long line inside it: !JOB ... !RUN MYPROG << data containing about 100 characters >> ... I built this file in EDITOR with LENGTH=132, RIGHT=132; the stream file looks great, with the full 100 characters in it. But, when I submit it, MYPROG sees only the first 80 characters! Help me! A: I'll bet you that, when you run SYSINFO (a really great program that you can usually find in PRV.TELESUP), and type )IO 10 << or whatever your :STREAMS device is configured at >> you'll see something like this: LDN DRT UN CH TY ST TYPE SPEED RECL DRIVER CLASSES 10 38 0 0 24 0 (7970) 40 HIOTAPE0 JOBTAPE See that RECL number? Its value is 40, and it means "record length". The record length of your :STREAMS device is configured at 40 words, or 80 bytes. Now the :STREAMS device is a mighty perverse sort of thing. If you can believe it, it is a "virtual device" emulating a tape drive! The practical effect of this, though, is that all the "input spool files" that the system builds -- these are the temporary disc files that contain all the input for any :STREAMed jobs -- will be written with a maximum record length of 80 bytes (40 words). The solution is fairly simple. Reconfigure your device 10 to have record size 128 words (256 bytes), or some such figure; then, that will become the new maximum :STREAM file line length. In "Configuring the MPE Streaming Facility", p. 5-19 of the "System Operation and Resource Management Reference Manual" (JUL 84), you are told to configure it as a TAPE DRIVER or a CARD READER, specifying parameters that "duplicate the values for an actual card reader or tape drive". Well, the recommended record with for a card reader is 40 words (see Table 7-12); for a tape driver, it's 128 words (Table 7-13). That's probably why you had a maximum record width of 40 words on your system in the first place -- whoever configured it chose the maximum record width value for a card reader or a tape drive. This, incidentally, should indicate WHY the :STREAMS device emulates a tape drive. Actually, it emulates a tape drive or a card reader; the latter, especially, was the way that you submitted ALL jobs to a computer in the bad old days of punch cards. As interactive devices (i.e. terminals) became more and more popular, people started wanting to submit jobs from already logged-on sessions. What the :STREAM command actually does is it "punches" a virtual card deck -- which is actually an input spool file -- which is then "read" by the :STREAMS device. Thus, to be precise, your mistake was a rather obvious one. You tried to punch 100 characters onto a punch cards that's only 80 characters wide! Haven't you ever used a REAL computer before? Q: I know that when you do a chained read on an IMAGE database, IMAGE can sometimes return to you an error 18 (BROKEN CHAIN) even though the database is structurally sound. I understand that this happens when somebody deletes the record that immediately follows the one I just read -- when I go to get the next record, IMAGE complains. However, recently I've been getting a very strange condition. I do a DBFIND and then some DBGET mode 5s; but, instead of getting me the records in the chain (which is what I want) or giving me an error 18 (which I can handle), it gets me records from an entirely different chain, with an entirely different key! What's happening? A: Each IMAGE chain is a DOUBLY-LINKED LIST. Every record in the chain points to the NEXT record in the chain and also to the PREVIOUS record in the chain. Say that you do a DBFIND with key "FOO" and then a DBGET mode 5. IMAGE knows: you've just gotten record number 17 -- the next record in the chain is number 27, and no previous record exists, since this is the first record in the chain. You process the record, and then go to DBGET the next one. But, hold on! While you were messing around with record 17, somebody else got to record 27 and deleted it. What's more, they've also DBPUT a new record, with key "BAR", that just happened to drop into the newly-vacated record number 27. Remember, any physical record in a detail dataset can end up having any key; it's simply a matter of what record slot was free at the time a DBPUT is done. In fact, it's quite likely that if you do a DBDELETE and then a DBPUT, the new record will be put into the same physical record slot that the just-deleted record occupied. Now, you do a DBGET mode 5. IMAGE says: "I know what the next record in the chain is -- it's record 27". Actually, that's what the next record in the chain WAS when the previous DBGET was done, but IMAGE doesn't realize it's changed. So, it gets record 27, unaware that the record now belongs to a different chain! Now, don't you start shouting "Eureka!" IMAGE's not THAT dumb. Sure, it gets record 27, BUT IT DOESN'T JUST RETURN IT TO YOU. First, it checks for precisely the circumstance that I just described -- that the record now belongs to a different chain. How can it do this? Well, it could save the current key value -- the one that we passed to DBFIND -- and then check every record it DBGETs (mode 5) to make sure that it has that key value. However, the key could be hundreds or even thousands of bytes long; IMAGE doesn't want to have to use all that memory, especially since there could be hundreds of users simultaneously accessing the database, each with his own key. What does IMAGE do? Well, when it gets record 27, it looks at the just-gotten record's PREVIOUS RECORD NUMBER. In a properly-structured doubly-linked list, the previous record number of the next record must point to the current record, right? Since record 27 was pointed to by the "next record number" field of record 17, the "previous record number" field of record 27 must point back to record 17. Now, record 17 is part of chain "FOO"; the only way record 27 can point back to record 17 is if it's part of the same chain. If record 27 has been deleted and a record with key "BAR" has been added in its place, then the new record 27 will point back to the previous entry in chain "BAR" -- not record 17. So, IMAGE checks the previous record number of the next record; if it doesn't point back to the current record, you get an IMAGE error 18. Not very nice, but certainly better than getting a record with the wrong key. (Note: An IMAGE error 18 can mean either a "true broken chain", which means that the links in the database are actually WRONG and the database is thus partially corrupted; or, it could mean -- as it does in this case -- a "temporary broken chain" condition, in which no real data corruption exists.) However, let's say that you read record 17 and then somebody else deletes BOTH records 17 AND 27, and then adds two more records BELONGING TO THE SAME CHAIN that just happen to fall back into records 17 and 27. Now, it's quite possible that the new record 27 points back to the new record 17 -- after all, they're on the same chain. This isn't the chain that the old record 17 was on, but IMAGE doesn't know this. All it knows is that the doubly-linked list is sound -- record 17 points to record 27, and record 27 points back to record 17. Since it doesn't check the keys, it doesn't detect the "broken chain" condition, so you get condition code 0 (all's well) and a record with key "BAR", even though you THOUGHT you were reading down the chain for key "FOO". If this strikes you as improbable, you're right. It's unlikely that, just as you're processing one record, somebody deletes both that record and the next one in the chain, and then adds new records with the same key that happen to fall into the same newly-vacated record slots. The very fact that it's improbable convinced the authors of IMAGE that they shouldn't bother checking for it; but one shouldn't confuse the IMPROBABLE with the IMPOSSIBLE. In fact, there's yet another circumstance in which a DBGET mode 5 could get you a record from the wrong chain. Say that you just did a DBFIND and are about to do your FIRST mode 5 DBGET. All that IMAGE knows is that the master record points to record 17, so that's the record it should get. But in the meantime (after the DBFIND but before the DBGET), somebody deleted record 17 and then added a new record (with a different key) that happened to fall into the same record 17. Now IMAGE goes to get record 17, and does its usual "previous record of next record must point to current record" check. But, in case of the first record in a chain, this check is somewhat different; the previous record pointer of the first record in a chain is always 0. When record 17 belonged to our "FOO" chain, this was true; however, it's still true even if record 17 is the first record of the "BAR" chain! Somebody deleted the old record and added the new record, but the previous record number is still 0, as it should be. There's no way (short of checking the key, which IMAGE doesn't do) for IMAGE to find out that it's now on the wrong chain. To summarize, there are two conditions in which IMAGE may return you the WRONG RECORD when you're doing a mode 5 DBGET: * If BOTH the current record and the next record in the chain have been deleted (between the previous DBGET and the current one), AND new records belonging to the same chain were put in their place. * If this is your first mode 5 DBGET after a DBFIND, and the first record in the chain has been deleted and a new record has been added in its place (as long as the new record is the first record of its own chain). In these cases, IMAGE won't signal a broken chain error; it'll just pretend that everything's OK, but return a record from the wrong chain. What's more, all the subsequent DBGET mode 5s will keep on following this erroneous chain. There are two possible solutions to this problem: * You might lock the dataset or at least the chain that you're traversing to make sure that nobody deletes any of its records while you're in the middle of reading it. It doesn't matter that you're only reading the chain, not modifying anything; somebody else's modifications might screw up your reads. * Or, you can keep track of the "current key" that you expect all the gotten records to have -- this is the same key you passed to the DBFIND -- and check each newly-gotten record to make sure that it has that key. If it doesn't, you should treat the situation the same way that you would a "broken chain" error -- either print an error message and abort (hoping that this problem doesn't happen very often) or going back to the beginning of the chain with a DBFIND and re-reading the entire chain from the beginning. This requires a great deal of extra logic (especially since you'll now be reading some records twice -- once on the first pass, and once on the second pass after the "broken chain" was detected). However, if you can't live with the DBLOCKs and aren't willing to just print an error message, you have to do this. Q: I'm running a program with its $STDIN redirected to a message file. Now, although my message file has variable length records (in fact, it seems that all message files have variable length records), the program seems to be getting all its input records with trailing blanks! What gives? A: Variable record length files are rather nifty things. I use them all the time for my job streams -- after all, a job stream is supposed to emulate a terminal, which is a variable-length input device. If you use a fixed record length file for a job stream, then all the programs that the stream runs will see their input with trailing blanks; some programs can't handle this. This is, of course, also the reason that you make your ;STDIN= file have variable length records -- if you're going to emulate the terminal, you might as well emulate it as closely as possible. Plus, of course, you save disc space, and as you noted, all message files must have variable length records anyway. So, variable record length files are good -- but why is the program getting all these trailing blanks? Well, this is caused by a little-known bit of behavior on the part of the file system, behavior that I think is quite inconsistent with the way things normally work. The FOPEN intrinsic has 13 parameters -- filename, foptions, aoptions, device, number of extents, etc. When you use FOPEN to create a new file, you might specify many of these parameters; you can tell the file system how many extents the file should have, what its filecode should be, whether it should have fixed-length records or variable-length records, etc. Now, say you're opening an already existing file. You might tell FOPEN its file size, its number of extents, its file code, its ASCII/BINARY flag, etc.; but what can the file system do with it? If the file already has filecode 123, 16 extents, and room for 1500 records, that's what it'll always have -- an FOPEN call can't change it. Now, there are still a lot of FOPEN parameters that make sense for an old file -- the access mode (INPUT/OUTPUT/APPEND/etc.), the number of buffers, etc. However, the file size, number of extents, number of user labels, blocking factor, etc. are all ignored when you're opening an old file. Normally, the record format (fixed, variable, or undefined) is one of those parameters that is ignored for already-existing files. If a file is built with fixed-length records, then opening it as a variable record length file won't change its fundamental structure; the file will remain a fixed record length file, and when read will always return records of the same length. Similarly, a variable record length file will always look like a variable record length file, even if it is opened as a fixed record length file (which, in fact, is the default FOPEN parameter setting). As you've probably guessed by now, message files are an exception. You :LISTF one, and you'll see it as a "VAM" or a "VBM" file (Variable Ascii/Binary Message file). But, when your program opens it, it will see it is as EITHER A FIXED RECORD LENGTH FILE OR AS A VARIABLE RECORD LENGTH FILE, DEPENDING ON THE SETTING GIVEN IN THE FOPEN CALL (which is bits .(8:2) of the "foptions" parameter). On disc, the message file has variable-length records -- but, if the program opens it with foptions.(8:2) = 0 (indicating fixed-length records), it will see all these records as fixed-length records, padded with trailing blanks if necessary. So, your program opens its $STDIN file (or has its $STDIN opened for it by MPE) with the default foptions, which indicate fixed-length records, and all subsequent reads of this file get fixed length records padded with blanks. All you need to do is override this with a :FILE equation, to wit :FILE INFILE=MSGFILE;REC=,,V :RUN MYPROG;STDIN=*INFILE Or, if you're running MYPROG from within a program (using the CREATEPROCESS intrinsic), you can specify the "REC=,,V" directly in the $STDIN file parameter (which can be a complete right-hand side of a file equation, not just a filename). And that's the story. Usually, the record format, the record size, the ASCII/BINARY flag, the file code, the number of extents, etc. are all ignored when opening an already existing file; however, when opening an already existing message file (whether it's the program's ;STDIN= or not), the record format (fixed vs. variable) is not ignored. Although the message file on disc remains variable record length, the program will see the file as fixed record length if that's the way it opened it. I suppose that the reason for this is the very fact that you're not allowed to :BUILD a message file with fixed length records; if you should happen to have a program that MUST have fixed length records for input, there has to be some way of viewing the message file as a fixed record length file. Q: When one of my programs reads its data file, it gets a rather bizarre error, as if the input data was incorrect. The data file looks OK, but a friend suggested that it might have some kind of "garbage characters" in it (like escapes or nulls). How can I find out if this is in fact the case? (I tried DISPLAY FUNCTIONS but it didn't show anything.) A: Well, DISPLAY FUNCTIONS shows many special characters but by no means all. In particular, it will not show: NULL (decimal 0) ENQ, also known as control-E (decimal 5) DEL (decimal 127) Also, any characters with the high-order (parity) bit set -- those with values 128-255 -- might be shown as either their parity-less equivalents (i.e. character 193 might be shown as A, which is a 65). Things are seldom what they seem. A file which seems to contain XYZ123 might actually have three null characters before the "X", a control-E between the "Z" and the "1", and the parity bit set in the "2" and the "3". Your program will see these characters, and will no doubt abort because of them; however, when you look at the file yourself using, say, EDITOR, you won't see them, even using DISPLAY FUNCTIONS. Your best bet is the FCOPY ;CHAR parameter. If you say :FCOPY FROM=DATAFILE;TO;CHAR then FCOPY will print all the records in DATAFILE, but replacing all garbage characters by "."s. If you say :FCOPY FROM=DATAFILE;TO;CHAR;OCTAL it will also display the data in each record in octal -- this way, you can see the exact values of each of the special characters (and also see which "."s are garbage characters and which are genuine dots!). You can even say :FCOPY FROM=DATAFILE;TO;CHAR;HEX which will display the data both as text (with dots replacing garbage characters) and in (surprise!) hexadecimal. Hex is actually better than octal for this, since in hex each byte is exactly two hex digits. Finally, if you don't want to use FCOPY and are fortunate (?) enough to have an HP 2645 terminal, you can hit the DISPLAY FUNCTIONS key and the CONTROL key simultaneously and enter so-called "MONITOR MODE" (some HP 2645s don't have this option, but most do). In this mode, all control characters, including NULLs, ENQs, and DELs will be displayed; however, characters with parity bit set will still be displayed as their parity-less equivalents, and every so often the terminal will hang up waiting for you to type a control-F (since in monitor mode even the ENQ/ACK protocol is displayed). On the plus side, hitting control-DISPLAY FUNCTIONS will actually make the DISPLAY FUNCTIONS light blink -- how high-tech! Q: Where are the HP system intrinsics kept? How are they different from, say, compiler library routines (like the mathematical routines such as SQRT, EXP, etc. or others like EXTIN', INEXT', and RAND)? How are they different from the "internal" routines like ATTACHIO and EXCHANGEDB? A: The actual machine code for all HP intrinsics is kept in the system SL -- SL.PUB.SYS. This, incidentally, is also where all the compiler library routines are kept, too, as are the internal MPE routines you mentioned. In fact, when we speak of the "operating system", we're really talking about SL.PUB.SYS (plus some other stand-alone PUB.SYS programs like LOAD, MEMLOG, and the I/O drivers). From MPE's point of view, then, FOPEN (an intrinsic), SQRT, and ATTACHIO are the same sort of thing -- system SL procedures that are called as external references of programs. If your program calls FOPEN, SQRT, and ATTACHIO, the loader will take care of these external references in exactly the same way -- it will find all three procedures in the system SL and link them to your program for the duration of its execution. The difference between FOPEN and ATTACHIO -- beyond, of course, the fact that they do different things -- is that you can say INTRINSIC FOPEN; << SPL >> SYSTEM INTRINSIC FOPEN << FORTRAN >> FUNCTION FOPEN: SHORTINT; INTRINSIC; << PASCAL >> CALL INTRINSIC "FOPEN" ... << COBOL II >> If you say this, then the compiler will simply KNOW WHAT THE NUMBER AND TYPE OF EACH PROCEDURE PARAMETER WILL BE. You can call ATTACHIO from SPL, FORTRAN, PASCAL, and COBOL, but the compiler won't know anything about the procedure parameters -- you'll have to define them yourselves (using an EXTERNAL declaration in SPL or PASCAL or by specifying the correct calling sequence in FORTRAN or COBOL), and woe to you if you define them incorrectly! The various compiler INTRINSIC constructs simply go to a file called SPLINTR.PUB.SYS and pick up the procedure definitions from there. SPLINTR.PUB.SYS (in its own rather cryptic way) defines, for instance, that FOPEN is OPTION VARIABLE and has 13 parameters, of which the first, fifth, and sixth are byte arrays passed by reference, the tenth is a double integer passed by value, and all the others are ordinary integer passed by value. This way, any compiler to which FOPEN is declared as an intrinsic will know how to generate the correct code to call it. Thus, if I were asked "what is an intrinsic?", I'd say "anything whose parameter layout is recorded in SPLINTR.PUB.SYS", which means the same as "anything which can be called using the INTRINSIC construct in SPL, FORTRAN, PASCAL, or COBOL II". Note that this includes more than what is documented in the System Intrinsics Manual -- it also includes all the IMAGE intrinsics, V/3000 intrinsics, graphics intrinsics, everything described in the Compiler Library Manual (including SQRT, EXTIN', INEXT', RAND, etc.), and other various and sundry routines. Thus, saying that something is an "intrinsic" is not really a reflection on where the procedure code resides (which is in SL.PUB.SYS, along with all the other non-intrinsics) or what the procedure does (except insofar as HP didn't choose to describe most of the more privileged system internal routines in the SPLINTR file). Intrinsics are just easier to call (from most languages) than non-intrinsics, since the language will "know" a lot about how the code to call the intrinsic needs to be constructed. Finally, to complicate matters further, SPL and PASCAL allow you to define your own "intrinsic files". Again, when you make your own procedure an "intrinsic", you aren't adding it to the operating system or even necessarily to the system SL (it could be in another USL, an RL, or a group or account SL) -- you're simply adding a description of its parameters to your own intrinsic file, thus making it simpler for your programs to call it. Q: How can I write a random number generator? I tried just calling the TIMER intrinsic and taking its results modulo some number, but the results didn't seem very random. Is there a random-number generator provided by HP? A: Writing computer games on company time, eh? Well, I can think of worse things to be doing. The HP Compiler Library Reference Manual (part number 30000-90028) is probably one of the least-read (and least-updated) manuals put out by HP. It describes exciting procedures like FMTINIT', FTNAUX', ADDCVRV', and such. However, it also describes: * A number of mathematical support routines (like cosine, sine, tangent, arctangent, logarithm, etc.); * Two rather nifty procedures called EXTIN' and INEXT', which convert strings to numbers and vice versa (including real numbers, in either exponential [1.234E-02] or fixed-point [.01234] format); * And, among other things, the HP random number generator. I guess the random number generator is considered part of the "compiler library" because compiled BASIC and FORTRAN programs might need it. In any case, it's provided by HP and even documented. There are two routines provided: * RAND1, which returns a real "seed" value. * RAND, which returns a real random number in the range 0.0 to just below 1.0 (i.e. RAND might return 0.0, but never 1.0). You pass it as a by-reference parameter the seed returned by RAND1; RAND modifies the seed on every call -- you should always pass the same seed variable to RAND. A reasonable SPL procedure that uses these routines might be: INTEGER PROCEDURE RANDOM (MAX); VALUE MAX; INTEGER MAX; BEGIN << Returns a random integer from 0 to MAX-1. >> OWN LOGICAL INITIALIZED:=FALSE; OWN REAL SEED; IF NOT INITIALIZED THEN BEGIN SEED:=RAND1; INITIALIZED:=TRUE; END; RANDOM:=INTEGER(FIXT(RAND(SEED)*REAL(MAX))); END; Note that the seed value passed to RAND completely determines the generated random number. Thus, if you wanted to, you could have the program input the seed value from the user rather than calling RAND1 to initialize it -- that way, you'll always be able to duplicate the sequence of random numbers generated by just specifying the same seed next time you run the program.

Go to Adager's index of technical papers