Release Notes: Earthworm Version "v6.1"

NEW MODULES:

==== The "Urban Hazards" project =====
 
The objective of the Earthworm Urban Hazards Project ("UH") is to support 
experimental deployments by the Urban Hazards Project at the USGS, Golden, CO. 
These experiments consist of deploying K2s in the field and periodically 
collecting .evt files from them. The desire is to associate the triggers from 
the .evt files with events from a known catalog. 

We've produced five modules to support this:

evtK2ora:
Reads .evt files, gets the station location from the DBMS, gets event
hypocenters from the DBMS, and tries to associate the trace data with events.
If an event associates with the trace data from an .evt file, that trace data is
inserted into the DBMS as a snippet belonging to that event. If all goes well, 
the .evt file is moved to a target directory.  Otherwise, the file
is moved into an error directory. 

cnsscat2ora:
Reads the supplied event list in CNSS format, and inserts it into the DBMS.

stalist_maker:
Reads through a set of .evt files, and constructs a station list based on the
stuff found in the header.  The station name is taken from the 'stationID'
field.  The channel number becomes the component name.  The network code is read
from the configuration file.  The location comes from the header.
	It can also accept such a station list as
input.  If one is supplied, any values found in this station list will override
the corresponding values found in the headers of the .evt files.  This permits a 
human to override erroneous values in the headers.  If all goes well, 
the .evt file is moved from the source directory into the target directory.  
Otherwise, the file is moved into the error directory. 

	statlist_loader:
Loads the above station list into the DBMS.

==== End "Urban Hazards" project =====


Export_scn_pri:
Export_scn_pri supports prioritized sending of waveform data.
It adds an optional parameter for the priority as the last element
of the Send_scn command. Valid values are 1 (highest) through 9 (lowest).
If any other value is found, or if the value is missing, then the default (9)
will be used.    

seiputaway: British Geological Survey (BGS) routine to write output data files in the 
SEISAN format.  This is now an option in module trig2disk. CJB 2/21/2002

gseputaway: BGS routine to write output files in GSE format. 
This is now an option in module trig2disk.  CJB 2/21/2002

findwave: Son of sniffwave. To snoop on a ring and write a file with a 
list of all SCN's found, plus some statistics. Alex 2/22/2


MODIFICATIONS/BUG FIXES TO EXISTING MODULES:
*******************************************
adsend: When told to get time from the system clock, adsend used to
use time(). This gave time to the nearest whole second, causing
weird time stamps. Changed to use _ftime(), which gives milliseconds.
ftime() may be NT specific, but so's adsend...
Alex 1/15/2

coaxtoring:  Made minor logging change.  Now, when a gap in packet
numbers is detected, coaxtoring logs the packet number of the first
packet after the gap.  WMK 8/10/01

wave_viewer:
************
KNOWN BUGS (wave_viewer)
1)  Wave_viewer often crashes during shutdown.  This is a harmless
    bug that only occurs when you close wave_viewer.  Sorry for the
    annoyance/inconvenience.

RELEASE v2.02  01/16/02  DK
Fixed several bugs, added several options.
1)  Added ScreenDelay and Queue (length) to the config file.
2)  Fixed bug in auto-scroll, that caused the display to scroll
    0.5% too fast.
3)  Fixed bug in Go-Oldest/Go-Youngest logic that was causing the
    display to go to the youngest/oldest data in the tank for a previous
    time, such as when the program was started.
4)  Added support for the accelerator bar (aka THUMBTRACK, aka draggin little
    box in the scrollbar at bottom of screen).  Support was accidentaly disabled
    in v2.00.

RELEASE v2.00  11/12/01
Rewrite of wave_viewer.  Please upgrade as older versions will not
be supported.
Rewrote core logic.  Now supports 1hz data, 200hz data, server 
reconnects, and other new features.  Fixes MANY bugs from older versions 
and is much less fragile.  Please see the README.txt file in the wave_viewer
directory.
DavidK 11/12/2001


wave_serverV: 
Changed to call tport_copyfrom instead of tport_getmsg.
Return codes from tport_copyfrom differentiate sequence gaps caused
by msgs never making it into the ring (GET_MISS_SEQGAP) from sequence gaps 
caused by msgs being overwritten before they're read from the 
ring (GET_MISS_LAPPED).  This allows us to get a better handle on why
gaps may appear in the wave_server tanks.  LDD 8/10/01

Changed to provide more accurate ASCII data replies, especially when
timestamps are sloppy.  Please see 
src/archiving/wave_serverV/README.changes for more detailed information.  
Also changed code that locates time offsets in the tanks, so that it 
accomodates timestamp slop.  Please see the README.changes file for more
info.
Davidk 08/11/01

sm_redi2ew:
Changed to read the eventid from the filename convention: EVENTID_ground.wave.
LDD 8/29/01

sm_tremor2ew:
Change to read the Origin Time line (2nd line) completely. Tremor version2
had added notes at the end of this line which we weren't expecting.
LDD 8/29/01

shakemapfeed:
Fixed a bug in new_sta where pA and pS were not being properly
managed. This resulted in bad XML code being passed to ShakeMap
which could not process the resulting file.
Lucky 9/26/2001

k2ew:
Updated the k2ew version from v2.25 to v2.26.  Now, if the program receives
a pcdrv_raw_read error or a pcdrv_raw_write error from the K2, it will send a
message to statmgr.  Pcdrv errors indicate that the flash memory in the K2
is bad.  Reformatting the flash memory may fix the problem.  Otherwise, the
flash memory may need to be replaced.  WMK 10/19/01

k2ew:
Updated the k2ew version from v2.26 to v2.27.  Changed type of variables
g_mt_working and g_ot_working to "volatile".  WMK 1/17/02

naqs2ew:
Improved its ability to recognize and reject packets with bogus timestamps.
Naqs2ew will not allow any time overlaps into the Earthworm system.  If it 
finds a time gap, it checks the packet timestamp against the system clock.
If the packet time is later than the clock by more than a given number
of seconds (set with the new command "TimeJumpTolerance"), then naqs2ew
assumes the timestamp is bogus and it ignores the packet.  This check 
against the system clock can be disabled by setting the tolerance to -1.
These changes were inspired by a hardware bug in the Nanometrics digitizers
that results in random large (days/years) single-packet time tears and also
random small (<1.0s) inter-packet time tears.  Nanometrics does have a fix
for the hardware bug, but it may take a while to change out some already
deployed units.  LDD 10/23/2001

hyp2000: Incorporated Fred Klein's most recent code.  This has
expanded arrays to account for additional component codes. This change
will be transparent to a normal user.  The source files he changed are 
common.inc, hycmd.for, hybda.for and hystl.for.  LDD 10/25/2001

gaplist: Added new optional command, Label.  This command can be interleaved
with the Scn commands to improve the readability of the gap table. All
Scn and Labels are printed in the table in the order they're listed in
the configuration file.  Label takes a character string (up to 30 chars)
as its only argument.  The string must be enclosed in double-quotes if it
contains embedded spaces.  LDD 12/07/2001

eqcoda: Fixed bug in function cdxtrp that resulted in very long
extrapolated codas being reported as negative values.  These negative
values would then overflow the archive message field and make eqcoda's
output unusable.  The archive-generating function was also changed to
weight-out any extrapolated coda lengths that would overflow the field,
and write them as 9999.  LDD 12/12/2001

sudsputaway: Changed sudsputaway to clip trace data values larger than long to
+/-2147483648 CJB 1/8/2002

localmag: Added code to perform a 5% cosine taper in time domain to each
end of the trace data. These tapers are additional data in front of the 
configured trace start time and after the trace end time. That is, the
portion of the trace that is not tapered remains as configured. This taper
will remove the possibility of `wrap-around' spikes in the pre-event
noise check window that sometimes appeared particularly with small events.
PNL 1/24/2002
 
sniffwave: Now prints the installationId and moduleId of each tracebuf
packet at the end of the tracebuf summary line.  LDD 2/1/2002

localmag: Added checks for short traces, such as might come from wave_server
for events with origin time close to current time. PNL 2/4/2002

localmag: Added check for too-long SAC trace. PNL 2/16/2002

putaway: Added code to call BGS seisan and gse format putaway routines.
 CJB 2/21/2002

gmew: The function gma() uses the wrong sign when integrating acceleration to
velocity and velocity to displacement. This has no effect in gmew since that
program reports absolute values for PGV and PGD. But when gma() is used
to obtain velocity and displacement time series, the error was noticed.
To support its use in other modules, gma.c has been moved from the gmew
directory to libsrc/utils; gma.h has been moved to the main include directory.
PNL 2/27/02

pick_ew: Made some minor debug logging changes. LDD 3/5/2002

localmag: Several changes that require a new config file.
1. Now when localmag runs as an earthworm module connected to transport
  ring, it will wait (sleep) until the computed length of trace has a chance
  to propagate to a hypothetical station at the maximum distance (set by 
  maxDist command). A new command extraDelay specifies a number of seconds
  to add to this computed wait time. The wait time is computed at startup;
  when a new event is received, localmag will log the number of seconds it
  will wait before processing the event.
2. Localmag uses new rules to compute the trace times and search
   times. Unchanged is the trace start: a specified number of seconds before
   the first P arrival from the layered velocity model.  Trace end is now a
   specified number of seconds after the Sg arrival computed using a specified
   Sg speed instead of the layered model.  Taper times are added to each end
   of the trace to get the 10% taper length as before.  Search start is now a
   fixed number of seconds before the selected search start phase - either
   first P or first S arrival from the layered model.  Search end is now a
   specified number of seconds after the Sg arrival computed using a specified
   Sg speed.  The Sg arrival is determined using hypocentral distance to the
   station.
3. The rules for deciding whehter a channel should be included in the
   magnitude for a station are changed. Before, channels would be used only if
   both the N and E components had amplitudes (eliminating traces with clipped
   or gappy data). Now, any E or N channel with a peak amplitude will be
   included in the station average. This makes the magnitude determination
   consistent with the magnitude message: at already contained single channels
   which confusingly were not included the station or event average magnitude
   before.
4. The LogFile command has been added to the config file; it works the same as
   in most other earthworm modules. The login_init calls were adjusted to the
   new standard (one call before reading config file with LogFile set to 1;
   then a second call after reading the config file to change LogFile to what
   the config file called for. That way errors found in the config file get
   logged instead of being discarded.)
   A few debug logit calls were adjusted to provide more useful info.
   PNL, 3/17/2002

reboot_mss_ew:  The program now checks to make sure the value of Logout in the
configuration file is either 0 or 1.  If not, the program exits shortly after
startup.  WMK  3/19/2002

heli_ewII:  Fixed bug where heli_ewII did not produce heartbeat messages while 
in the startup or update gifs phases.  JMP 03/20/2002

tracebuf.h: The size of the net string was given as TRACE_CHAN_LEN, the size 
of the chan string as TRACE_NET_LEN. While these two macros currently have
the same value, they may not in the future. This bug fix has no impact on
current operations. PNL 3/20/2002

ora2snippet_gif:  Added code to de-mean the waveforms that are displayed
in the gif.  Added code to allow for several decimation/drawing schemes
for decimating data and displaying it in the GIF.  Fixed a bug where
the return codes from WaveMsgMakeLocal() were not being handled and the
program was core dumping when it tried to parse bad snippets.
DK 03/20/2002
In WiggleAlign == ALIGN_WIGGLE_SHOW_ALL mode, added code to calculate 
epicentral distances (via geo_to_km()) for stations reporting trace.  
Added code to order snippets by distance from epicenter.
DK 03/21/2002

swap.c:  WaveMsgMakeLocal() was modified to include a checksum like 
calculation, that checks the header before converting the data contents
of a tracebuf message.  This was done because of core-dump problems 
in ora2snippet_gif with partially corrupted headers.
WaveMsgMakeLocal() also now returns an error when either _INTEL or
_SPARC is not defined. It also returns an error when a corrupted 
header is detected.
DK 03/20/2002

CHANGES TO CONFIGURATION FILES and DESCRIPTOR FILES:
**************************************************** 
naqs2ew.d:
New command "TimeJumpTolerance" (in seconds) is used to catch packets with 
dubious timestamps.  If a time gap is detected in an incoming data stream, 
the new packet's timestamp is compared to the system clock.  If it is later 
than the current system time by more than TimeJumpTolerance seconds, 
naqs2ew assumes the packet timestamp is bogus (it's in the future) and it 
ignores the entire packet. You could also think of this term as a measure
of how close to UTC you expect your system clock to be. And beware, if
you use this feature with small tolerances, the PC's system clock must be 
kept pretty close to network time, or you may end up throwing away good
data. Set to -1 if you never want to compare packet times to the system 
clock.  Valid values are -1 or any number = 0.0
Example: TimeJumpTolerance 600.
LDD 10/23/2001

gaplist.d:
New Optional command, Label, can be interleaved with Scn commands.
Examples:  Label "Ash Creek channels 1-6"
           Label CampSix
LDD 12/07/2001

trig2disk.d:
New output file formats - seisan and gse_int  CJB 2/21/2002

waveman2disk.d:
New output file formats - seisan and gse_int  CJB 2/21/2002

localmag.d and localmag_uw.d:
New required command SgSpeed
New optional commands extraDelay, searchStartPhase, searchTimes and LoggFile.
Removed optional command searchWindow.  PNL, 3/17/2002

ora2snippet_gif.d:
Added "WiggleType" command, that allows selection of a 
decimation/drawing method.  There are currently 4 to choose
from.
Added "Debug" command for turning debugging on.
DK 03/20/2002


KNOWN BUGS:
***********
k2ew: 
k2ew uses the k2hdr.rwParms.misc.channel_bitmap parameter to decide
which channels it will see as streams from the K2.  This is actually the
parameter which the K2 uses to decide which channels to record in an event
file (see K2's  command).  The K2's  is the parameter
that shows which channels the K2 is streaming out.  This is the parameter
that k2ew should be using to decide what data it will see.  The big problem
is that none of the header files from Kinemetrics (nkwhdrs.h) seem to 
show this parameter anywhere. This issue is only a problem for those folks
who want to record more channels on the K2 than they want to stream continuously.
Terry Dye (Univ of Utah) discovered this problem when trying to stream only
3 channels of a 12 channel K2.  LDD 2002/04/11

k2ew:
handle leakage problem. When K2ew can't establish a connection to the K2, it keeps
trying, but seems to leak handles on NT. Alex 4/16/02

wave_serverV:
it occasionally sends the following error to statmgr:
   UTC_Thu Sep 06 03:30:14 2001  WSV1/wave_serverV_nano ReadBlockData
   failed for tank [z:\nano57.tnk], with offset[999908] and record
   size[64]! errno[0] Mail sent.
The nominal tank size is 1 megabyte, and the actual tank size is 999908.
It looks like waveserver is trying to read off the end of the tank.
WMK 9/6/01

wave_serverV:
appears not to reply to requests for a single sample of data. 
I noticed when testing wave_viewer, that if the start time and end time of a 
request were equal (in which case there should be one sample of data)
then wave_serverV did not reply to the request (ASCII request) at all.	
No Data, No Flags, No Reply, No Nothing.  It needs to issue a reply to every request.
DavidK 09/25/01

Wave_viewer often crashes during shutdown.  This is a harmless
    bug that only occurs when you close wave_viewer.  Sorry for the
    annoyance/inconvenience.

Adsend:
Automatic restarts of adsend (using the "restartMe" line in the descriptor
file) can cause an NT system to hang. Therefore, you should never
use the autorestart feature with adsend, but you should bring down
the entire Earthworm system if adsend needs to be restarted.
LDD 5/31/2000 Comments added to adsend.desc, but leave this warning here!

libsrc/utils/site.c: 
The strings used for station, channel and network are
required to be fixed length with trailing spaces added to short names. If
the strings given to site_index do not have these trailing blanks, SCN names
will not match. This is not documented anywhere.  PNL 10/15/00

sm_ew2ora:
There is a bug in sm_ew2ora that involves having multipe time intervals for
components and channels.  If a strong motion message containing information
for a channel that the DB has never seen before is loaded into the DB, and
then later another message for the same channel with an earlier timestamp
is loaded, the load of the second message will fail, due to problems with
overlapping time intervals, and the call that sm_ew2ora uses to create
those time intervals.  This problem only affects stations that were
not previously loaded via one of the station loader programs stalist*2ora,
and only when receiving SM data that is timestamped with a time that is
prior to the original time for that channel.  The bug lies in the logic
of ewdb_api_PutSMMessage(), and not in the underlying code.
Davidk 05/25/01

A change was made to ewdb_api_PutSMMessage() that dramatically affects
sm_ew2ora.  Please see the note about that function.  Davidk 2001/07/26 


KNOWS DEFICIENCIES:
*******************

ringtocoax:
In Windows NT, the time resolution of sleep_ew() is about 16 msec (one clock
tick).  On Solaris, the resolution is about 10 msec.  This is a problem for 
ringtocoax, since packet delays need to be set to a few milliseconds.

statmgr: 
A space is needed between "tsec:" and the value. 
If it isn't there, things fail. Artifcat of the kom routines. (Alex)

threads functions: 
The KillThread function on WindowsNT and Solaris
terminate the thread without ensuring that no mutexes are held. If a thread
holds a mutex when it dies, no other thread can get that mutex. PNL 1/12/2000

carlsubtrig:
The system time must be set to GMT and ew_nt.cmd must have 
TZ=GMT for carlsubtrig to work.  Comments in ew_nt.cmd done 5/25/00. Barbara
	
localmag:
needs GMT set on the system

ew2seisvole:
on NT, exits with horrible crash when system is stopped.

NUMBER OF RINGS LIMITED ON SOLARIS:
Under Solaris 2.6 (and probably other versions as well), the maximum number 
of shared memory segments is six. This means that on an out-of-the-box machine
you can only configure six rings. If you try to configure more than that, you
will see a cryptic message from tport_create about too many open files.  The
fix to this problem is to add the following lines to the /etc/system
file, and then reboot the system.

 set shmsys:shminfo_shmmax = 4294967295
 set shmsys:shminfo_shmmin = 1
 set shmsys:shminfo_shmmni = 100
 set shmsys:shminfo_shmseg = 20
 set semsys:seminfo_semmns = 200
 set semsys:seminfo_semmni = 70

This allows for 20 rings.

     Lucky Vidmar (7/6/2000)


startstop_solaris: 
There MAY be a problem with the signal that 
startstop sends to modules during the shutdown sequence. The shutdown 
sequence is started (after typing "quit" to startstop or running "pau")
by startstop placing a terminate message on all transport rings. Modules
should see this message and start their own shutdown. After a configurable
delay, startstop checks to see that all modules have exitted. Any that are 
still running are sent a signal to terminate them. Currently that signal
is SIG_TERM. But since wave_serverV has a handler for SIG_TERM, wave_serverV
sees that as essentially the same as a terminate message. So if wave_serverV
is having problems completing its shutdown, SIG_TERM won't do anything. The
result is that startstop may give up and exit, leaving wave_serverV running.
If that happens, the operator will have to terminate wave_serverV by doing
"kill -9 ". That may leave shared memory and semaphores
stranded in the kernel: run the command "ipcs -a" to see. If necessary,
the stranded shared memory and semaphores may be cleaned up with the
ipcrc command; must be run as root; see the man page.
This problem only exists on Solaris/Unix, not on WindowsNT.
PNL, 10/4/2000

libsrc/utils/kom.c:  fix comment in k_open()

The comment above k_open() says that only one file can
be open at a time. Yet the Kbuf array has slots for MAXBUF (currently 4) open
files. Does this work, or is the comment to be taken at it's word?
PNL 10/15/00

libsrc/utils/logit.c: 
logit_init() requires a module_id number, which it uses
to construct the log file name. This is not helpful, since the module_id
number is not meaningful to people. Worse, it requires that the config file be
read and earthworm.d lookups be completed before logit calls can be made. Thus
errors in the config file can only be reported to stderr or stdout instead of
being saved in a file.  PNL 11/29/00

TRACEBUF messages.
The definition of `endtime' of the TRACEBUF message is not documented.
Some programmers are taking it as the "expected start time of the next
TRACEBUF packet (if the sample interval is uniform.)" The more accepted
practice is that `endtime' is the time of the last sample of the current
TRACEBUF packet; that is, one sample interval less than the expected
start time of the next TRACEBUF messsage. Using this last definition, if a
TRACEBUF packet has exactly one sample, then its starttime and endtime are
the identical. Clearly this distinction needs to be documented. The file
waveform_format (in the /home/earthworm/DOC directory) gives no specifics 
about start or end times.  PNL 1/24/01
 

Date last modified: April 16, 2002