ENVIRONMENT VARIABLES USED BY THE RAP LIBRARIES

This document describes the most commonly-used environment variables which control the functionality of the RAP libraries.

NOTE: When a variable may be set to "true" or "false", this is case-insensitive.

DS SERVER SYSTEM LATEST DATA INFO PROCESS MAPPER JUNEAU DATA MAPPER SERVER MAPPER RSL IMAGES MDV GRIDS

DS SERVER SYSTEM

The DS server system is the latest server layer for DIDSS (Data ingest and distributed server system).

DS Server System

Variable Default value Description Library

RAP_DATA_DIR undefined If set, URLs point to locations relative to RAP_DATA_DIR, unless the file part of the URL starts with / or ., in which case the absolute or relative paths are used. didss dsserver

DATA_DIR undefined Alternartive to RAP_DATA_DIR. If RAP_DATA_DIR is defined, it is used. If not, DATA_DIR is used if defined. didss dsserver

DATA_MAPPER_ACTIVE true Set to 'false' to turn off automatic access from clients to DataMapper. dsserver

DATA_MAPPER_DEBUG undefined Set to 'true' to see debug messages from DATA_MAPPER access requests. dsserver

DS_COMM_TIMEOUT_MSECS 30000 This is the timeout, in milli-secs, used in communications between clients and servers. The default value of 30 secs (30000 msecs) was chosen so that programs would not block too long and fail to register with procmap, thereby causing them to be restarted. You may need to inccrease the value over slow lines. If you increase the value, you may be wise to run without the restart layer. dsserver

DS_PING_TIMEOUT_MSECS 10000 This is the timeout, in milli-secs, used when pinging a server to see if it is alive. It is used by clients to check whether a server is up, before making the data request. If the server is down, the client makes a request to the server manager (DsServerMgr) to start the server. The DsServerMgr starts the server and then uses a ping to make sure it is alive, before returning a successful flag to the client. dsserver

DS_BASE_PORT 5430 All ports in the DS server system are calculated relative to this port. If you need to run two server systems on a single host, for eaxmple for two different users, you can override the default. To be safe, pick a number at least 1000 above the default, because the server ports are in the immediate range above the base. dsserver

SPDB_ALLOW_NO_LOCK undefined If set to "true", the Spdb library will not require a lock on the data base files for reads. Locks are still required for writes. This may be used if you are reading data across a cross-mount for which file locking is not implemented. However, the better strategy is to contact a server which has local access to the data. Spdb

CLOSE_SOCKET_IN_CHILD undefined If set to "true", the servers will close the listening socket in child processes. This should not be necessary, but was for Linux kernels 2.0.x, and possibly earlier kernels. It is not necessary on Solaris or Linux potato or kernels 2.4 and later. Only set this variable if your system is getting too many open files. You can check this with the 'lsof' command. dsserver

MAX_FORECAST_LEAD_DAYS 10 When seraching a directory for data stored in forecast file name format (yyyymmdd/g_hhmmss/f_ssssssss.ext) the library needs to know how far back in time to look for forecast data which may be valid at the current time. It looks back a maximum of this number of days. didss

MDV_WRITE_FORMAT Set the write format for MDV apps Options are:

FORMAT_MDV (the default)

FORMAT_XML (XML header and data buffer)

FORMAT_NCF (NetCDF CF)

Mdv

MDV_WRITE_USING_EXTENDED_PATHS Tells MDV apps to write using extended file paths Options are:

TRUE

FALSE (the default)

If TRUE, extended paths will be written. For example:
yyyymmdd/hhmmss.mdv
becomes
yyyymmdd/yyyymmdd_hhmmss.mdv
Mdv

MDV_WRITE_ADD_YEAR_SUBDIR Tells MDV apps to prepend the year to the output path Options are:

TRUE

FALSE (the default)

If TRUE, the year will be added to the output path. For example:
yyyymmdd/hhmmss.mdv
becomes
yyyy/yyyymmdd/hhmmss.mdv
Mdv

FCOPY_SERVER_ALLOW_NO_LOCK Allow DsFCopyServer to not lock files on write. Options are:

TRUE

FALSE or missing (the default)

If TRUE, lock files will not be required on write. Normally locking should be used. Only set this to allow writing across a cross-mount, which is not recommended anyway. But if you have to .... dsserver

FCOPY_SERVER_TMP_DIR Specify tmp dir for DsFCopyServer on write. Options are:

Specified-tmp-path

missing (the default)

Normally DsFCopyServer writes tmp files in the subdirectory being used. If this env var is set, the tmp file will be written to the specified directory. WARNING - make sure the tmp dir and the data dir are on the same partition. dsserver

USER Account user name The Ds Server messaging layer uses this to tag each message with the client's user name. You can spoof the user name by resetting this variable. dsserver

DS Server System
Variable	Default value	Description	Library
RAP_DATA_DIR	undefined	If set, URLs point to locations relative to RAP_DATA_DIR, unless the file part of the URL starts with / or ., in which case the absolute or relative paths are used.	didss dsserver
DATA_DIR	undefined	Alternartive to RAP_DATA_DIR. If RAP_DATA_DIR is defined, it is used. If not, DATA_DIR is used if defined.	didss dsserver
DATA_MAPPER_ACTIVE	true	Set to 'false' to turn off automatic access from clients to DataMapper.	dsserver
DATA_MAPPER_DEBUG	undefined	Set to 'true' to see debug messages from DATA_MAPPER access requests.	dsserver
DS_COMM_TIMEOUT_MSECS	30000	This is the timeout, in milli-secs, used in communications between clients and servers. The default value of 30 secs (30000 msecs) was chosen so that programs would not block too long and fail to register with procmap, thereby causing them to be restarted. You may need to inccrease the value over slow lines. If you increase the value, you may be wise to run without the restart layer.	dsserver
DS_PING_TIMEOUT_MSECS	10000	This is the timeout, in milli-secs, used when pinging a server to see if it is alive. It is used by clients to check whether a server is up, before making the data request. If the server is down, the client makes a request to the server manager (DsServerMgr) to start the server. The DsServerMgr starts the server and then uses a ping to make sure it is alive, before returning a successful flag to the client.	dsserver
DS_BASE_PORT	5430	All ports in the DS server system are calculated relative to this port. If you need to run two server systems on a single host, for eaxmple for two different users, you can override the default. To be safe, pick a number at least 1000 above the default, because the server ports are in the immediate range above the base.	dsserver
SPDB_ALLOW_NO_LOCK	undefined	If set to "true", the Spdb library will not require a lock on the data base files for reads. Locks are still required for writes. This may be used if you are reading data across a cross-mount for which file locking is not implemented. However, the better strategy is to contact a server which has local access to the data.	Spdb
CLOSE_SOCKET_IN_CHILD	undefined	If set to "true", the servers will close the listening socket in child processes. This should not be necessary, but was for Linux kernels 2.0.x, and possibly earlier kernels. It is not necessary on Solaris or Linux potato or kernels 2.4 and later. Only set this variable if your system is getting too many open files. You can check this with the 'lsof' command.	dsserver
MAX_FORECAST_LEAD_DAYS	10	When seraching a directory for data stored in forecast file name format (yyyymmdd/g_hhmmss/f_ssssssss.ext) the library needs to know how far back in time to look for forecast data which may be valid at the current time. It looks back a maximum of this number of days.	didss
MDV_WRITE_FORMAT	Set the write format for MDV apps	Options are: FORMAT_MDV (the default) FORMAT_XML (XML header and data buffer) FORMAT_NCF (NetCDF CF)	Mdv
MDV_WRITE_USING_EXTENDED_PATHS	Tells MDV apps to write using extended file paths	Options are: TRUE FALSE (the default) If TRUE, extended paths will be written. For example: yyyymmdd/hhmmss.mdv becomes yyyymmdd/yyyymmdd_hhmmss.mdv	Mdv
MDV_WRITE_ADD_YEAR_SUBDIR	Tells MDV apps to prepend the year to the output path	Options are: TRUE FALSE (the default) If TRUE, the year will be added to the output path. For example: yyyymmdd/hhmmss.mdv becomes yyyy/yyyymmdd/hhmmss.mdv	Mdv
FCOPY_SERVER_ALLOW_NO_LOCK	Allow DsFCopyServer to not lock files on write.	Options are: TRUE FALSE or missing (the default) If TRUE, lock files will not be required on write. Normally locking should be used. Only set this to allow writing across a cross-mount, which is not recommended anyway. But if you have to ....	dsserver
FCOPY_SERVER_TMP_DIR	Specify tmp dir for DsFCopyServer on write.	Options are: Specified-tmp-path missing (the default) Normally DsFCopyServer writes tmp files in the subdirectory being used. If this env var is set, the tmp file will be written to the specified directory. WARNING - make sure the tmp dir and the data dir are on the same partition.	dsserver
USER	Account user name	The Ds Server messaging layer uses this to tag each message with the client's user name. You can spoof the user name by resetting this variable.	dsserver

LATEST DATA INFO FILES

The latest data info files are small text files which are written to a data directory to indicate the time of the latest data written to that directory. The default file name is _latest_data_info. There is also a file message queue (FMQ) option which causes the latest info also to be written to an FMQ.

Latest Data Info files

Variable Default value Description Library

LDATA_FMQ_ACTIVE true By default the FMQ is active. If this is set to "false", the FMQ (file message queue) option will be deactivated. If active, the routines write both a text file and an FMQ containing the latest data info. The FMQ option is useful for cases in which data arrives rapidly and the client may miss data information while polling. Since the FMQ is a queue, the client can read the entries without risk of missing an entry. didss

LDATA_FMQ_NSLOTS 2500 The number of slots in the FMQ, if it is active. The queue wraps once this number of entries is reached. Increase if data arrives very rapidly and the clients may need more time to read the queue. didss

LDATA_NO_WRITE undefined If set to "true", no _latest_data_info files will be written. This is sometimes useful in archive mode when you do not want to overwrite the realtime files. toolsa didss dsserver

Latest Data Info files
Variable	Default value	Description	Library
LDATA_FMQ_ACTIVE	true	By default the FMQ is active. If this is set to "false", the FMQ (file message queue) option will be deactivated. If active, the routines write both a text file and an FMQ containing the latest data info. The FMQ option is useful for cases in which data arrives rapidly and the client may miss data information while polling. Since the FMQ is a queue, the client can read the entries without risk of missing an entry.	didss
LDATA_FMQ_NSLOTS	2500	The number of slots in the FMQ, if it is active. The queue wraps once this number of entries is reached. Increase if data arrives very rapidly and the clients may need more time to read the queue.	didss
LDATA_NO_WRITE	undefined	If set to "true", no _latest_data_info files will be written. This is sometimes useful in archive mode when you do not want to overwrite the realtime files.	toolsa didss dsserver

PROCESS MAPPER

The process management layer relies on 'procmap', the process mapper. Programs register a hearbeat with procmap, typically once per minute. Normally one procmap runs per host. The auto_restart script checks the procmap list against an expected list of processes, and restarts those which are not running.

procmap

Variable Default value Description Library

PROCMAP_HOST localhost The host name on which the process mapper is running. Running programs register with the mapper running on this host. For most systems this will be the local host. There are almost no systems in use which use a central process mapper. toolsa

PROCMAP_HOST2 undefined Alternative process mapper host, for redundant systems. If set, running programs register with the mapper running on this host in addition to the main one. For most systems this is not used. toolsa

PROCMAP_DIR undefined If defined, it should be a directory path. Each time a process registers with procmap, the status string will be written to a file in this directory. This is useful for debugging if a system crashes badly and procmap no longer runs. It is seldom used. toolsa

PROCMAP_VERBOSE undefined If set to "true", verbose debugging messages are printed out during communication with procmap. toolsa

USER Account user name When the programs contact procmap, they use this variable to indicate the name of the user running the process. You can spoof the user name by resetting this variable. toolsa

procmap
Variable	Default value	Description	Library
PROCMAP_HOST	localhost	The host name on which the process mapper is running. Running programs register with the mapper running on this host. For most systems this will be the local host. There are almost no systems in use which use a central process mapper.	toolsa
PROCMAP_HOST2	undefined	Alternative process mapper host, for redundant systems. If set, running programs register with the mapper running on this host in addition to the main one. For most systems this is not used.	toolsa
PROCMAP_DIR	undefined	If defined, it should be a directory path. Each time a process registers with procmap, the status string will be written to a file in this directory. This is useful for debugging if a system crashes badly and procmap no longer runs. It is seldom used.	toolsa
PROCMAP_VERBOSE	undefined	If set to "true", verbose debugging messages are printed out during communication with procmap.	toolsa
USER	Account user name	When the programs contact procmap, they use this variable to indicate the name of the user running the process. You can spoof the user name by resetting this variable.	toolsa

JUNEAU DATA MAPPER

The Juneau data management layer relies on 'datamap', the Juneau data mapper. As programs write data, they register with datamap.

datamap

Variable Default value Description Library

DATAMAP_HOST localhost The host name on which the Juneau data mapper is running. For most systems this will be the local host. toolsa

DATAMAP_HOST2 undefined Alternative Juneau data mapper host, for redundant systems. If set, programs register data information with the mapper running on this alternative host in addition to the main one. For most systems this is not used. toolsa

DATAMAP_VERBOSE undefined If set to "true", verbose debugging messages are printed out during communication with datamap. toolsa

datamap
Variable	Default value	Description	Library
DATAMAP_HOST	localhost	The host name on which the Juneau data mapper is running. For most systems this will be the local host.	toolsa
DATAMAP_HOST2	undefined	Alternative Juneau data mapper host, for redundant systems. If set, programs register data information with the mapper running on this alternative host in addition to the main one. For most systems this is not used.	toolsa
DATAMAP_VERBOSE	undefined	If set to "true", verbose debugging messages are printed out during communication with datamap.	toolsa

SERVER MAPPER

NOTE: The server mapper is part of the old server infrastructure, and is no longer in widespread use.

servmap

Variable Default value Description Library

SERVMAP_HOST localhost The host name on which the server mapper is running. Servers register with the mapper running on this host. For most systems this will be the local host. toolsa

SERVMAP_HOST2 undefined Alternative server mapper host, for redundant systems. If set, servers register with the mapper running on this host in addition to the main one. For most systems this is not used. toolsa

SERVMAP_VERBOSE undefined If set to "true", verbose debugging messages are printed out during communication with servmap. toolsa

servmap
Variable	Default value	Description	Library
SERVMAP_HOST	localhost	The host name on which the server mapper is running. Servers register with the mapper running on this host. For most systems this will be the local host.	toolsa
SERVMAP_HOST2	undefined	Alternative server mapper host, for redundant systems. If set, servers register with the mapper running on this host in addition to the main one. For most systems this is not used.	toolsa
SERVMAP_VERBOSE	undefined	If set to "true", verbose debugging messages are printed out during communication with servmap.	toolsa

RSL RADAR DATA LIBRARY

The RSL radar data handling library was written by NASA for the TRMM project.

rsl library

Variable Default value Description Library

RSL_COLOR_DIR undefined The directory for color tables used in RSL image rendering routines. trmm_rsl

rsl library
Variable	Default value	Description	Library
RSL_COLOR_DIR	undefined	The directory for color tables used in RSL image rendering routines.	trmm_rsl

MDV GRID CLASS

Mdv grid class

Variable Default value Description Library

MDV_GRID_TOLERANCE 0.0000001 Tolerance used when checking whether two grids are equal. mdv