Lib-DVM - detailed design (contents) Part 1(1-7) Part 2 (8-11) Part 3 (12) (Error messages)
document date: february, 2001 - last edited 03.05.01 -

8 Output of informational messages of Run-Time System and error messages

8.1 Functions of informational and error messages output

Each message, reported by Run-Time System, is either informational or error message.

Informational messages are output by the functions rtl_printf, rtl_mprintf (see section 1.3), and by the function pprint with parameter prefix, equal to 0 or 1 (see section 7.3).

Moreover, for informational messages output Run-Time System provides the function rtl_iprintf that is syntactically equivalent to the function printf and returns the number of output symbols:

int rtl_iprintf (char *format, . . . );

Any trace information, output to informational message stream by tprintf and dvm_trace functions (see section 7.3) is informational message too.

Error message output is performed by eprintf and epprintf functions, and also by the function pprint with prefix parameter, equal to 2 or 3 (see section 7.3).

8.2 Controlling standard output streams and streams of informational and error message output

The direction and redirection of the streams, intended for output of information and error messages, is performed by the parameter current.par and syspar.* files.

The file current.par is the parameter file of current startup of Run-Time System and is created by Run-Time System startup program in the current working directory. The file consists of the following line sequence:

CurrentPar[0] = <parameter>;
CurrentPar[1] = <parameter>;
. . . . . . . . . . . . . . . . . . . . . . . .
CurrentPar[n] = <parameter>;

The parameters, placed from the right of "=" and determining information output, can be:

1. +o[<regime>][<path>[.<extension>]]

The parameter instructs Run-Time system to redirect stdout stream to the file (files with name (names).

[<path>][<processor number>].<extension> .

If <extension> option is not specified, the extension of the file (files) is "sto".

The <regime> option defines a way to redirect stdout stream and can be "*" or "+". In the first case the redirection is performed at all the processors to its own for every processor file with the name above. In the second case the redirection is also performed at all the processors but to the common for all the processors file with the name.

<path>.<extension> .

At that if <path> option (together with <extension> option) is not specified, the redirection is performed to the file with "stdout" name (without extension).

If <regime> option is omitted, the redirection of stdout stream is performed only at the processor with zero internal number to the file <path>.<extension> (or to "stdout" file, if <path> option is not specified).

2. +e[<regime>][<path>[.<extension>]]

The parameter specifies the redirection of stderr stream to file (files) with a name (names)

[<path>][<processor number>].<extension> .

The redirection of stderr stream is simularly to stdout stream redirection, considered above but extension of the file (files) is "ste" by default, and the file name is "stderr" if <path> option is omitted.

3. +i[<symbols, determining streams>]

This parameter instructs Run-Time System to redirect informational messages to the streams, determined by specified symbols.

The symbols, determining the streams, can be:
"o" - stdout stream;
"e" - stderr stream;
"f" - file.

If both symbols "o" and "e" are specified and stdout and stderr streams are not redirected to the files or redirected to the same file, informational messages will be output to stdout stream only (informational messages are not duplicated). When informational messages are directed to the file (symbol "f" is specified) the file name (path) is defined by SysInfoFileName parameter in parameter syspar.* files (see below). If no symbols are specified informational messages will not be output.

Redirection of stdout and stderr streams to the files can be done using following parameters, specified in syspar.* files:

StdOutToFile - flag to redirect standard output stream stdout into the file (files);
StdOutFileName - parameter that defines redirection mode and a name (names) of a file (files) for redirection of stdout stream. The parameter has the form:
[<regime>][<path>[.<extension>]] ;
the parameter options and redirection scheme are similarly to options and scheme considered above when describing "+o" parameter of current.par file;
StdErrToFile - flag to redirect standard error output stream stderr to the file (files);
StdErrFileName - parameter that defines redirection mode and a name (names) of a file (files) for redirection of stderr stream. The parameter has the form:
[<regime>][<path>[.<extension>]] ;
the parameter options and redirection scheme are similarly to options and scheme considered above when describing "+e" parameter of current.par file;
DelStdStream - flag to delete "old" (kept from previous startup) files containing output streams.

When parameters are output and processed the first specification of standard output stream redirection to the file has the highest priority. Partially stdout and stderr streams redirection using "+o" and "+e" parameters of current.par file suppresses the stream redirection by listed above parameters of syspar.* files (the file current.par is output and processed earlier than syspar.* file).

If the instruction "+i" is omitted in the file current.par the informational message output is determined by the parameters, specified in syspar.* file.

The parameters are:

SysInfoPrint - common flag of enabled output of informational messages;
SysInfoStdOut - flag to output informational messages to stdout stream provided that it is redirected to the file;
SysInfoStdErr - flag to output informational messages to stderr stream;
SysInfoFile - flag to output informational messages to the file.

If value of the parameters SysInfoStdOut and SysInfoStdErr is equal to 1 then informational messages are also redirected to the file and are output to the screen if stdout is redirected to the file (redirection of a user program output to the file must be specified by "+o" parameter of current.par file). Controlling values of the parameters any combination of SCREEN - STDOUT, REDIRECTED TO THE FILE can be obtained. If SysInfoStdOut and SysInfoStdErr parameters are equal to one, and stdout and stderr streams are not redirected or redirected to the same file informational messages will be output to stdout stream only (informational messages are not duplicated).

Note, that the redirection of informational messages using the current.par parameter "+i" has priority under the redirection, specified by the parameters SysInfoStdOut, SysInfoStdErr and SysInfoFile of syspar.* file.

Output of informational messages to the file (SysInfoFile = 1) is also controlled by next parameters of syspar.* file:

SysInfoFileName - path, specifying a file for informational messages accumulating;
FatInfoNoOpen - flag to terminate the work in the case of opening failure of the file for the informational messages;
SysInfoSepFile - flag to accumulate informational messages in separate for each processor file;
DelSysInfo - flag to delete “old” (kept from previous startup) files containing informational messages.

If FatInfoNoOpen parameter is equal to zero opening failure of informational message file causes warning output to the error message stream and then the program will run without output of informational messages to the file.

If SysInfoSepFile is equal to 0, each processor outputs informational messages directly to the file, specified by SysInfoFileName parameter. If SysInfoSepFile is equal to 1, each processor accumulates informational messages in its own temporary file, having name <internal processor number>.out. After work completion all temporary files are jointed in the one file (specified by parameter SysInfoFileName), and then are deleted.

All error messages are:

Therefore error messages always:

Note 1. Output of informational messages before input of SysInfoPrint parameter value is determined by state of common flag of informational messages output at program startup (before Run-Time System initialization). The flag is assigned by non-zero value in the case if Run-Time System was compiled with predefined compilation variable _INIT_INFO_. Otherwise informational messages are not output up to input of SysInfoPrint parameter.

Note 2. Usually output of informational messages is performed as follows. Up to input of parameters informational messages are directed to the stdout stream. After SysInfoStdOut and SysInfoStdErr parameters input if the fist parameter is equal to zero but the second one is not, informational messages will be redirected to stderr stream. If both parameters are not equal to zero, and stdout and stderr streams are redirected to the screen or to the same file, informational message will be output to the stdout stream only.
Informational message output with stderr stream priority is provided as alternative. For this purpose Run-Time System should be compiled with defined compilation variable _INFO_STDERR_. As result stderr stream will be play the same role as stdout stream in standard scheme considered above.

Note 3. If informational message output is set to the file and to stdout (stderr) stream, but stdout (stderr) stream is also redirected to this file, output of informational messages to the file is locked (SysInfoFile parameter is set to zero).

8.3 Classes of informational messages and their output controlling

All informational messages are divided on non-crossed classes; each of them is corresponded to functioning of certain component of Run-Time System. Output of each class messages is controlled independently by listed below parameters from syspar.*, statist.*, systrace.* and usrdebug.* files. The parameter is considered to be specified in syspar.* files by default.

Messages of any class are output only in the case if common flag of informational message output is set (using SysInfoPrint parameter, considered in section 8.2).

9 Measuring time of user program interval execution

In addition to main tools of performance analysis and gathering information, required for it, Run-Time System provides subsidiary mechanism of measuring execution time of user program intervals. When this mechanism is used the results of measuring are output to system trace streams. To mark beginning and end of measured interval macrocalls MeasureStart and MeasureFinish are used. Measured intervals can be nested:

MeasureStart; /* interval 1 beginning */
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  MeasureStart; /* interval 2 beginning */
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 
 
    MeasureStart; /* interval N beginning /
. . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . .
MeasureFinish; / interval N end */
   
   
   
  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 
  MeasureFinish; /* interval 2 end */
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 
MeasureFinish; /* interval 1 end */

The macrocall MeasureStart is equivalent to the function dvm_trace call (see section 7.3) with event number 1 (the event name is MeasureStart), and macrocall MeasureFinish is equivalent to dvm_trace function call with event number 2 (the event name is MeasureFinish). As additional information for the macrocall MeasureStart the enclosure level of measured interval (1, 2, … , N in the scheme above) is output and for the macrocall MeasureFinish the enclosure level and program execution time, passed from MeasureStart macrocall moment with the same enclosure level, are output to the opened streams.

Note that trace events with numbers 1 and 2 can be disabled as any other events. That is alternative to exclusion of MeasureStart and MeasureFinish macrocalls from the program.

10 Requesting global and local sizes of specified dimension of specified object

long GetObjectSize ( ObjectRef
unsigned long
ObjRef,
Axis );

The function GetObjectSize returns size of the object specified by reference ObjRef for the dimension Axis (global size or the size of non-distributed object).

The object can be:

The size of parallel loop dimension is equal to

ceil ( (LastIndex - InitIndex + 1)/Step )

where:

InitIndex - initial value of index variable of parallel loop for dimension Axis;
LastIndex - last value of index variable of parallel loop for the dimension Axis;
Step - step of index variable of parallel loop for the dimension Axis;
ceil - function to find the least integer, not less then a value of its argument.

If Axis equal to 0, the size of whole object (product of sizes of all dimensions) is returned.

If ObjRef is not reference to any object above, the function returns zero.

long GetObjectLocSize ( ObjectRef
long
ObjRef,
Axis );

The function GetObjectLocSize is similarly to GetObjectSize function considered above but returns local size of dimension Axis (or local size of the whole object) that is the size of the object part, mapped on the current processor. If the object is processor system, then the values, returned by GetObjectSize and GetObjectLocSize functions are equal.

Note that the local size of whole object having no local part on the current processor is equal to zero.

The functions GetObjectSize and GetObjectLocSize are not traced and not processed by statistics accumulation program, therefore they are faster, then getsiz_ and locsiz_ functions, considered in document rtsIDe.doc.

11 Tools to support Run-Time System built-in program of statistic information accumulation

11.1 Matrix of execution characteristics of the user program current interval

11.1.1 Primary execution characteristics of user program current interval

All Run-Time System functions, the user program can invoke, are subdivided on the following not crossed groups:

MsgPasGrp - message passing functions
StartRedGrp - functions to start reduction operations;
WaitRedGrp - functions to wait for reduction completion;
RedGrp - other functions to support reduction;
StartShdGrp - functions to start distributed array shadow edge renewing;
WaitShdGrp - functions to wait for shadow edge renewing completion;
ShdGrp - other functions to support shadow edge exchange;
DistrGrp - functions to distribute data over processors;
ReDistrGrp - functions to redistribute data over processors;
MapPLGrp - functions to support parallel loop execution;
DoPLGrp - functions to request next portion of parallel loop iterations for the portion execution (dopl_ function);
PrgBlockGrp - functions to support program blocks;
IOGrp - input/output functions;
RemAccessGrp - functions to support remote data access;
UserDebGrp - functions of dynamic control and tracing of user program execution (function of interaction with DVM-debugger);
StatistGrp - functions to support user program intervals, intended for gathering information, required to program performance analysis;
SystemGrp - other Run-Time System functions (not belonging to the groups, listed above).

The group names are defined as C named number constants in system.def file of Run-Time System source files. A number of function groups is defined by named constant StatGrpCount. The correspondence <function name> <group name> is defined in source file intergrp.c.

Execution characteristics of functions of every group are gathered in the current interval time consumption matrix IM, created when new current interval is created (interval matrix of characteristics, matrix of the current interval characteristics). When matrix IM is created, its pointer must be assigned to the global variable

s_GRPTIMES (* CurrInterPtr)[StatGrpCount]; .

After leaving the current interval statistics accumulation program must process contained in matrix IM data and restore in CurrInterPtr variable the pointer to the matrix of the interval, completed interval is nested in.

The matrix IM is squire matrix, which i-th row elements contain execution characteristics of the functions of i-th group, directly called by the user program. Each function of i-th group can invoke (directly or indirectly) the functions of any j-th group (i and j can be equal). Run-Time System accumulates characteristics of such calls in IM[i,j] element. So, i-th row of matrix IM contains decomposition of execution characteristics of i-th group functions, directly called by user program, by direct or indirect function calls of all groups.

IM[i,j] element is a structure of the type

typedef struct {


}
double
double
double
s_GRPTIMES;
CallCount;
ProductTime;
LostTime;

where:

CallCount - characteristic of a count of j-th group function calls by i-th group functions;
ProductTime - characteristic of total productive time of j-th group functions when they are called by i-th group functions;
LostTime - characteristic of total lost time of j-th group functions when they are called by i-th group functions.

When IM matrix is created, zero value must be assigned to all listed above characteristic in all elements of the matrix.

CallCount characteristic is increased by 1/n every time after exit from j-th group function (n is number of processors, executing current branch of the parallel program).

ProductTime and LostTime characteristics are increased by time/n and (n- 1)*time/n correspondently after exit from j-th group function (time is execution time of j-th group function not taking into account the time of the calls of other group functions, n is a number of processors, executing current branch).

On completion of the function (of i-th group), called directly by the user program, ProductTime and LostTime characteristics, contained in IM[i][i] element, and CallCount characteristic, contained in IM[UserGrp][i] element, are increased (about row with UserGrp number see below). It allows, first, to have total elapsed time of i-th group functions, called by the user program, in one row of IM matrix, and, second, to have information about amount of each group function calls, performed directly by the user program.

The meaning of characteristics, contained in elements of the row with UserGrp number, differs from the meaning of characteristics in other rows of matrix IM. ProductTime and LostTime characteristics of IM[UserGrp][UserGrp] element is productive and lost times of the user program itself (not taking into account the time of Run-Time System function calls). ProductTime and LostTime characteristics in all other elements of UserGrp row are equal to zero. CallCount characteristic of [UserGrp][i] element is zero, and for all other elements IM[UserGrp][j] (j UserGrp) contains average number of j-th group function calls from the user program per processor.

11.1.2 About characteristics of passing message function group

Besides the function group UserGrp, considered in section 11.1, program execution characteristics of the function group MsgPasGrp (message passing) also differ from the same named characteristics of other groups.

All execution time of any message passing function is considered as lost time, that is accumulated in LostTime characteristic in "pure" form, that is without correction, taking into account a number of processors, executing the current branch (without multiplication by (n-1)/n coefficient) The number of message passing function calls are also summarized in CallCount characteristic without division by n. In ProductTime characteristic time of real (actual) dissynchronization of the program is accumulated instead of average productive time per processor.

Real dissynchronization time is integral measure of productive time loses because of not simultaneous performing of message receiving by one processor and invoking the function, sending this message, by the other processor. To calculate this time, Run-Time System keeps astronomical time of message passing function call as auxiliary information, accompanying the message (origin astronomical time at all the processors is the same, see time synchronization in section 6). Depending on used message passing scheme (see section 5), the time of real dissynchronization is calculated at receiving side in the following way.

1. Synchronous message exchange.

Real dissynchronization time is equal to absolute value of difference between the time of the function rtl_Recv call by receiving side and the time of the function rtl_Send call by sending side.

2. Asynchronous message exchange.

Real dissynchronization time is equal to difference between time of the function rtl_SendA call by sending side and time of the function rtl_RecvA call by receiving side. If the difference is negative, real dissynchronization time is zero.

3. Message exchange in "NO WAIT" mode.

Real dissynchronization time is equal to difference between time of the function rtl_Sendnowait call by sending side and time of the function rtl_Waitrequest call by receiving side. If the difference is negative, real dissynchronization time is zero.

Total time of real dissynchronization is a sum of real dissynchronization times of separate interconnection actions <send-receive>.

11.1.3 Calculation of summary characteristics of program execution using interval matrix of characteristics

Denote the set of function group numbers as GNS:

GNS = { MsgPasGrp, StartRedGrp, WaitRedGrp, RedGrp, StartShdGrp, WaitShdGrp, ShdGrp, DistrGrp, ReDistrGrp, MapPLGrp, DoPLGrp, PrgBlockGrp, IOGrp, RemAccessGrp, UserDebGrp, StatistGrp, SystemGrp }

  1. Characteristic of own productive time of the program execution itself (not taking into account Run-Time System calls):

OUPT = IM[UserGrp][UserGrp].ProductTime

  1. Characteristic of own lost time of program execution itself (not taking into account Run-Time System calls):

OULT = IM[UserGrp][UserGrp].LostTime

  1. Characteristic of productive execution time of i-th group functions, directly called by the user program (i MsgPasGrp):

  1. Characteristic of productive execution time of Run-Time System functions, directly called by the user program:

  1. Characteristic of lost execution time of i-th group functions, directly called by the user program:

  1. Characteristic of lost execution time of Run-Time System functions, directly called by the user program:

  1. Characteristic of productive time of user program execution:

UPT = OUPT + SPT

  1. Characteristic of lost time of the user program execution:

ULT = OULT + SLT

  1. Characteristic of the number of Run-Time System function calls, performed directly by user program:

  1. Characteristic of real dissynchronization time when executing i-th group functions, directly called by user program:

RMTi = IM[i][MsgPasGrp].ProductTime

  1. Characteristic of real dissynchronization time of user program execution:

  1. Characteristic of own productive time of j-th group function execution (not taking into account other group function calls) (j MsgPasGrp):

  1. Characteristic of own time of j-th group function execution (not taking into account other group function calls):

11.1.4 Auxiliary matrix of user program characteristics

To control and debug information accumulation and visualization for performance analysis Run-Time System has auxiliary matrix of user program execution characteristics:

s_GRPTIMES TaskInter[StatGrpCount][StatGrpCount]; .

The matrix TaskInter corresponds to initial interval of the user program execution. The information accumulation is performed independently on main mechanism of statistics accumulation. Program execution characteristics, contained in TaskInter matrix elements, are not corrected by the number of processors, executing the current branch.

On program termination specified part of TaskInter matrix is output to Run-Time System informational message stream.

The parameters TimeExpendPrint and StatGrpName of statist.* files control information accumulation in TaskInter matrix. If TimeExpendPrint is equal to zero, the information accumulation in TaskInter matrix is not done. Non-zero value of the parameter defines accumulation mode and structure and form of output information.

1 - brief output mode; the following information is output: - total time of Run-Time System operating (System Time)
- time of task solving (Task Time) and included in it execution time of Run-Time System functions (Function Time) (without division on productive and lost times);
2 - in addition to brief mode the characteristics of the column, which is the sum of TaskInter matrix columns, is output (s_GRPTIMES structures are summarized by element when TaskInter matrix elements are summarized).
3 - in addition to brief mode the characteristics of the row, which is the sum of TaskInter matrix rows, is output;
4 - in addition to brief mode the characteristics of the column, which name-number is defined by StatGrpName parameter value;
5 - in addition to brief mode the characteristics of the row, which name-number is defined by StatGrpName parameter value;

11.1.5 Supporting characteristic accumulation of input/output DVM-operations implemented in Fortran

Usually when the user program calls any Run-Time System function, the function group number is saved in global variable

int CurrOper; .

During the called function execution and on its completion the value of CurrOper variable is used as matrix IM row number to keep measured characteristics.

When DVM input/output operations in Fortran are executed, Run-Time System functions that are not input/output ones are called. To correct accumulation of characteristics of Fortran input/output operations it is necessary:

To implement these requirements Run-Time System provides the functions

void biof_(void);

and

void eiof_(void); .

The user program at the beginning of input/output operation calls the function biof_. The function biof_ assigns IOGrp value to CurrOper variable and 1 to global variable

unsigned char CurrOperFix; .

If CurrOperFix variable has non-zero value, CurrOper value is not changed when the user program invokes Run-Time System functions.

The function eiof_ is intended to inform Run-Time System about input/output operation completion. Its execution consists of the assignment to CurrOperFix zero value, allowing updating CurrOper value.

11.2 Tools of external controlling statistics accumulation program

The external control of statistics accumulation program functioning is implemented by the following parameters, specified in parameter files statist.*:

Is_DVM_STAT - flag to enable statistics accumulation;
StatBufLength - size in bytes of statistics accumulation buffer for one processor;
StatFileName - name of the file for uploading the statistics accumulation buffers from all the processors, assigned to the task;
DelStatist - flag to delete "old" (kept from previous startup) file with accumulated trace.
IsSynchrTime - flag to count real dissynchronization time (flag to accumulate ProductTime characteristic in MsgPasGrp column of interval characteristic matrix IM);
IsTimeVariation - flag to count time of potential dissynchronization and time variation;
MaxIntervalLevel - maximal interval nesting level of the user program execution.

11.3 Initializing statistics accumulation program

When statistics accumulation program is initialized, the values are assigned to the following global variables (corresponding to the external parameters, considered in section 11.2):

char *StatBufPtr - pointer to the buffer of statistics information accumulation;
long StatBufLength - size in bytes of the buffer of statistics information accumulation.

After initialization of mentioned above global variables the function

void stat_init (void); (statist.c file)

is called to more specific statistics accumulation tuning.

In particular, stat_init function must give to Run-Time System initial interval characteristic matrix (must initialize CurrInterPtr pointer, see section 11.1.1).

After stat_init function return Run-Time System considers that statistics accumulation is activated.

11.4 Processing calls of Run-Time System functions by statistics accumulation program

After exiting stat_init function, considered in section 11.3, Run-Time System begins to accumulate characteristics of user program execution in interval matrix of characteristics.

The characteristics are counted in IM matrix on entry in and exit from any Run-Time System function, accessible to the user program. Entry in and exit from some Run-Time System functions require additional processing. Such functions are the functions, which call and exit times are required to calculate user program potential dissynchronization and time variation. When entering and/or leaving such function Run-Time System calls the function

void stat_event(int EventNumber); (statist.c file),

where EventNumber is occurred trace event number. The parameter EventNumber exactly defines as processed function as occurred event type (entry in processed function or exit from it).

The stat_event function is called by EventNumber event only if EventNumber-th element of global array

unsigned char IsStat[];

is non-zero. Non-zero (1-255) value of IsStat[EventNumber] byte defines the processing way of the event with EventNumber number by stat_event function. The array IsStat is initialized in Run-Time System module statevnt.c, where for entry in and exit from each function, accessible for user program, zero or non-zero value of corresponding element of IsStat array is set.

Moreover, stat_event function can be invoked only if the following requirements, common for all processed functions, are satisfied:

Before the function stat_event call Run-Time System assigns to the global variable

ObjectRef StatObjectRef;

reference to the current processed object. Such object dependently on occurred event, can be:

For a distributed array the zero-numbered word of its header is assigned to the variable StatObjectRef.

The reference to the current processed object kept in variable StatObjectRef, can be used by statistics accumulation program to obtain inquired information about object by Run-Time System functions.

Note, that prior to stat_event function call Run-Time System keeps current astronomical time in global variable

double Curr_dvm_time; .

So during stat_event function execution the current time request by dvm_time function is not needed.

11.5 Terminating statistics accumulation program

When terminating the user program Run-Time System invokes the function

void stat_done (void); (statist.c file)

for final processing gathered information by statistics accumulation program (if such processing is required). After exit from this function Run-Time System writes statistics accumulation buffers of all the processors into bit file with the name, defined by the StatFileName parameter of external control (see section 11.2). The processor buffers are placed in this file sequentially in the order of increasing internal numbers of the processors. Then it is considered statistics accumulation is disabled.

11.6 Access to some system information

Some characteristics of Run-Time System and its hardware-software environment may be useful and are contained in the following global variables.

long MPS_TYPE - message passing system type. It can have the values (named constants are given): EMP_MPS_TYPE, GNS_MPS_TYPE, ROU_MPS_TYPE, MPI_MPS_TYPE, PVM_MPS_TYPE.
long ProcCount - a number of processors, assigned to the task.
long CurrEnvProcCount - number of processors, performing current branch of parallel program.
unsigned char VMSRank - rank of processor system, where Run-Time System operates (rank of the initial processor system).
long VMSSize[] - array, which i-th element contains size of the (i+1)-th dimension of the initial processor system (0 = i = VMSRank-1).
int MPS_CurrentProc - internal number of the current processor.
int MPS_IOProc - internal number of input/output processor (for initial processor system).
int DVM_IOProc - internal number of input/output processor of the current processor system.
int MPS_CentralProc - internal number of central processor (for initial processor system).
int DVM_CentralProc - internal number of central processor of the current processor system.
double Curr_dvm_time - current time of Run-Time System (a result of last time requesting by dvm_time function).

Lib-DVM - detailed design (contents) Part 1(1-7) Part 2 (8-11) Part 3 (12) (Error messages)