Release Notes: Earthworm Version "v6.1"
==== The "Urban Hazards" project ===== The objective of the Earthworm Urban Hazards Project ("UH") is to support experimental deployments by the Urban Hazards Project at the USGS, Golden, CO. These experiments consist of deploying K2s in the field and periodically collecting .evt files from them. The desire is to associate the triggers from the .evt files with events from a known catalog. We've produced five modules to support this: evtK2ora: Reads .evt files, gets the station location from the DBMS, gets event hypocenters from the DBMS, and tries to associate the trace data with events. If an event associates with the trace data from an .evt file, that trace data is inserted into the DBMS as a snippet belonging to that event. If all goes well, the .evt file is moved to a target directory. Otherwise, the file is moved into an error directory. cnsscat2ora: Reads the supplied event list in CNSS format, and inserts it into the DBMS. stalist_maker: Reads through a set of .evt files, and constructs a station list based on the stuff found in the header. The station name is taken from the 'stationID' field. The channel number becomes the component name. The network code is read from the configuration file. The location comes from the header. It can also accept such a station list as input. If one is supplied, any values found in this station list will override the corresponding values found in the headers of the .evt files. This permits a human to override erroneous values in the headers. If all goes well, the .evt file is moved from the source directory into the target directory. Otherwise, the file is moved into the error directory. statlist_loader: Loads the above station list into the DBMS. ==== End "Urban Hazards" project ===== Export_scn_pri: Export_scn_pri supports prioritized sending of waveform data. It adds an optional parameter for the priority as the last element of the Send_scn command. Valid values are 1 (highest) through 9 (lowest). If any other value is found, or if the value is missing, then the default (9) will be used. seiputaway: British Geological Survey (BGS) routine to write output data files in the SEISAN format. This is now an option in module trig2disk. CJB 2/21/2002 gseputaway: BGS routine to write output files in GSE format. This is now an option in module trig2disk. CJB 2/21/2002 findwave: Son of sniffwave. To snoop on a ring and write a file with a list of all SCN's found, plus some statistics. Alex 2/22/2 MODIFICATIONS/BUG FIXES TO EXISTING MODULES: ******************************************* adsend: When told to get time from the system clock, adsend used to use time(). This gave time to the nearest whole second, causing weird time stamps. Changed to use _ftime(), which gives milliseconds. ftime() may be NT specific, but so's adsend... Alex 1/15/2 coaxtoring: Made minor logging change. Now, when a gap in packet numbers is detected, coaxtoring logs the packet number of the first packet after the gap. WMK 8/10/01 wave_viewer: ************ KNOWN BUGS (wave_viewer) 1) Wave_viewer often crashes during shutdown. This is a harmless bug that only occurs when you close wave_viewer. Sorry for the annoyance/inconvenience. RELEASE v2.02 01/16/02 DK Fixed several bugs, added several options. 1) Added ScreenDelay and Queue (length) to the config file. 2) Fixed bug in auto-scroll, that caused the display to scroll 0.5% too fast. 3) Fixed bug in Go-Oldest/Go-Youngest logic that was causing the display to go to the youngest/oldest data in the tank for a previous time, such as when the program was started. 4) Added support for the accelerator bar (aka THUMBTRACK, aka draggin little box in the scrollbar at bottom of screen). Support was accidentaly disabled in v2.00. RELEASE v2.00 11/12/01 Rewrite of wave_viewer. Please upgrade as older versions will not be supported. Rewrote core logic. Now supports 1hz data, 200hz data, server reconnects, and other new features. Fixes MANY bugs from older versions and is much less fragile. Please see the README.txt file in the wave_viewer directory. DavidK 11/12/2001 wave_serverV: Changed to call tport_copyfrom instead of tport_getmsg. Return codes from tport_copyfrom differentiate sequence gaps caused by msgs never making it into the ring (GET_MISS_SEQGAP) from sequence gaps caused by msgs being overwritten before they're read from the ring (GET_MISS_LAPPED). This allows us to get a better handle on why gaps may appear in the wave_server tanks. LDD 8/10/01 Changed to provide more accurate ASCII data replies, especially when timestamps are sloppy. Please see src/archiving/wave_serverV/README.changes for more detailed information. Also changed code that locates time offsets in the tanks, so that it accomodates timestamp slop. Please see the README.changes file for more info. Davidk 08/11/01 sm_redi2ew: Changed to read the eventid from the filename convention: EVENTID_ground.wave. LDD 8/29/01 sm_tremor2ew: Change to read the Origin Time line (2nd line) completely. Tremor version2 had added notes at the end of this line which we weren't expecting. LDD 8/29/01 shakemapfeed: Fixed a bug in new_sta where pA and pS were not being properly managed. This resulted in bad XML code being passed to ShakeMap which could not process the resulting file. Lucky 9/26/2001 k2ew: Updated the k2ew version from v2.25 to v2.26. Now, if the program receives a pcdrv_raw_read error or a pcdrv_raw_write error from the K2, it will send a message to statmgr. Pcdrv errors indicate that the flash memory in the K2 is bad. Reformatting the flash memory may fix the problem. Otherwise, the flash memory may need to be replaced. WMK 10/19/01 k2ew: Updated the k2ew version from v2.26 to v2.27. Changed type of variables g_mt_working and g_ot_working to "volatile". WMK 1/17/02 naqs2ew: Improved its ability to recognize and reject packets with bogus timestamps. Naqs2ew will not allow any time overlaps into the Earthworm system. If it finds a time gap, it checks the packet timestamp against the system clock. If the packet time is later than the clock by more than a given number of seconds (set with the new command "TimeJumpTolerance"), then naqs2ew assumes the timestamp is bogus and it ignores the packet. This check against the system clock can be disabled by setting the tolerance to -1. These changes were inspired by a hardware bug in the Nanometrics digitizers that results in random large (days/years) single-packet time tears and also random small (<1.0s) inter-packet time tears. Nanometrics does have a fix for the hardware bug, but it may take a while to change out some already deployed units. LDD 10/23/2001 hyp2000: Incorporated Fred Klein's most recent code. This has expanded arrays to account for additional component codes. This change will be transparent to a normal user. The source files he changed are common.inc, hycmd.for, hybda.for and hystl.for. LDD 10/25/2001 gaplist: Added new optional command, Label. This command can be interleaved with the Scn commands to improve the readability of the gap table. All Scn and Labels are printed in the table in the order they're listed in the configuration file. Label takes a character string (up to 30 chars) as its only argument. The string must be enclosed in double-quotes if it contains embedded spaces. LDD 12/07/2001 eqcoda: Fixed bug in function cdxtrp that resulted in very long extrapolated codas being reported as negative values. These negative values would then overflow the archive message field and make eqcoda's output unusable. The archive-generating function was also changed to weight-out any extrapolated coda lengths that would overflow the field, and write them as 9999. LDD 12/12/2001 sudsputaway: Changed sudsputaway to clip trace data values larger than long to +/-2147483648 CJB 1/8/2002 localmag: Added code to perform a 5% cosine taper in time domain to each end of the trace data. These tapers are additional data in front of the configured trace start time and after the trace end time. That is, the portion of the trace that is not tapered remains as configured. This taper will remove the possibility of `wrap-around' spikes in the pre-event noise check window that sometimes appeared particularly with small events. PNL 1/24/2002 sniffwave: Now prints the installationId and moduleId of each tracebuf packet at the end of the tracebuf summary line. LDD 2/1/2002 localmag: Added checks for short traces, such as might come from wave_server for events with origin time close to current time. PNL 2/4/2002 localmag: Added check for too-long SAC trace. PNL 2/16/2002 putaway: Added code to call BGS seisan and gse format putaway routines. CJB 2/21/2002 gmew: The function gma() uses the wrong sign when integrating acceleration to velocity and velocity to displacement. This has no effect in gmew since that program reports absolute values for PGV and PGD. But when gma() is used to obtain velocity and displacement time series, the error was noticed. To support its use in other modules, gma.c has been moved from the gmew directory to libsrc/utils; gma.h has been moved to the main include directory. PNL 2/27/02 pick_ew: Made some minor debug logging changes. LDD 3/5/2002 localmag: Several changes that require a new config file. 1. Now when localmag runs as an earthworm module connected to transport ring, it will wait (sleep) until the computed length of trace has a chance to propagate to a hypothetical station at the maximum distance (set by maxDist command). A new command extraDelay specifies a number of seconds to add to this computed wait time. The wait time is computed at startup; when a new event is received, localmag will log the number of seconds it will wait before processing the event. 2. Localmag uses new rules to compute the trace times and search times. Unchanged is the trace start: a specified number of seconds before the first P arrival from the layered velocity model. Trace end is now a specified number of seconds after the Sg arrival computed using a specified Sg speed instead of the layered model. Taper times are added to each end of the trace to get the 10% taper length as before. Search start is now a fixed number of seconds before the selected search start phase - either first P or first S arrival from the layered model. Search end is now a specified number of seconds after the Sg arrival computed using a specified Sg speed. The Sg arrival is determined using hypocentral distance to the station. 3. The rules for deciding whehter a channel should be included in the magnitude for a station are changed. Before, channels would be used only if both the N and E components had amplitudes (eliminating traces with clipped or gappy data). Now, any E or N channel with a peak amplitude will be included in the station average. This makes the magnitude determination consistent with the magnitude message: at already contained single channels which confusingly were not included the station or event average magnitude before. 4. The LogFile command has been added to the config file; it works the same as in most other earthworm modules. The login_init calls were adjusted to the new standard (one call before reading config file with LogFile set to 1; then a second call after reading the config file to change LogFile to what the config file called for. That way errors found in the config file get logged instead of being discarded.) A few debug logit calls were adjusted to provide more useful info. PNL, 3/17/2002 reboot_mss_ew: The program now checks to make sure the value of Logout in the configuration file is either 0 or 1. If not, the program exits shortly after startup. WMK 3/19/2002 heli_ewII: Fixed bug where heli_ewII did not produce heartbeat messages while in the startup or update gifs phases. JMP 03/20/2002 tracebuf.h: The size of the net string was given as TRACE_CHAN_LEN, the size of the chan string as TRACE_NET_LEN. While these two macros currently have the same value, they may not in the future. This bug fix has no impact on current operations. PNL 3/20/2002 ora2snippet_gif: Added code to de-mean the waveforms that are displayed in the gif. Added code to allow for several decimation/drawing schemes for decimating data and displaying it in the GIF. Fixed a bug where the return codes from WaveMsgMakeLocal() were not being handled and the program was core dumping when it tried to parse bad snippets. DK 03/20/2002 In WiggleAlign == ALIGN_WIGGLE_SHOW_ALL mode, added code to calculate epicentral distances (via geo_to_km()) for stations reporting trace. Added code to order snippets by distance from epicenter. DK 03/21/2002 swap.c: WaveMsgMakeLocal() was modified to include a checksum like calculation, that checks the header before converting the data contents of a tracebuf message. This was done because of core-dump problems in ora2snippet_gif with partially corrupted headers. WaveMsgMakeLocal() also now returns an error when either _INTEL or _SPARC is not defined. It also returns an error when a corrupted header is detected. DK 03/20/2002 CHANGES TO CONFIGURATION FILES and DESCRIPTOR FILES: **************************************************** naqs2ew.d: New command "TimeJumpTolerance" (in seconds) is used to catch packets with dubious timestamps. If a time gap is detected in an incoming data stream, the new packet's timestamp is compared to the system clock. If it is later than the current system time by more than TimeJumpTolerance seconds, naqs2ew assumes the packet timestamp is bogus (it's in the future) and it ignores the entire packet. You could also think of this term as a measure of how close to UTC you expect your system clock to be. And beware, if you use this feature with small tolerances, the PC's system clock must be kept pretty close to network time, or you may end up throwing away good data. Set to -1 if you never want to compare packet times to the system clock. Valid values are -1 or any number = 0.0 Example: TimeJumpTolerance 600. LDD 10/23/2001 gaplist.d: New Optional command, Label, can be interleaved with Scn commands. Examples: Label "Ash Creek channels 1-6" Label CampSix LDD 12/07/2001 trig2disk.d: New output file formats - seisan and gse_int CJB 2/21/2002 waveman2disk.d: New output file formats - seisan and gse_int CJB 2/21/2002 localmag.d and localmag_uw.d: New required command SgSpeed New optional commands extraDelay, searchStartPhase, searchTimes and LoggFile. Removed optional command searchWindow. PNL, 3/17/2002 ora2snippet_gif.d: Added "WiggleType" command, that allows selection of a decimation/drawing method. There are currently 4 to choose from. Added "Debug" command for turning debugging on. DK 03/20/2002 KNOWN BUGS: *********** k2ew: k2ew uses the k2hdr.rwParms.misc.channel_bitmap parameter to decide which channels it will see as streams from the K2. This is actually the parameter which the K2 uses to decide which channels to record in an event file (see K2'scommand). The K2's is the parameter that shows which channels the K2 is streaming out. This is the parameter that k2ew should be using to decide what data it will see. The big problem is that none of the header files from Kinemetrics (nkwhdrs.h) seem to show this parameter anywhere. This issue is only a problem for those folks who want to record more channels on the K2 than they want to stream continuously. Terry Dye (Univ of Utah) discovered this problem when trying to stream only 3 channels of a 12 channel K2. LDD 2002/04/11 k2ew: handle leakage problem. When K2ew can't establish a connection to the K2, it keeps trying, but seems to leak handles on NT. Alex 4/16/02 wave_serverV: it occasionally sends the following error to statmgr: UTC_Thu Sep 06 03:30:14 2001 WSV1/wave_serverV_nano ReadBlockData failed for tank [z:\nano57.tnk], with offset[999908] and record size[64]! errno[0] Mail sent. The nominal tank size is 1 megabyte, and the actual tank size is 999908. It looks like waveserver is trying to read off the end of the tank. WMK 9/6/01 wave_serverV: appears not to reply to requests for a single sample of data. I noticed when testing wave_viewer, that if the start time and end time of a request were equal (in which case there should be one sample of data) then wave_serverV did not reply to the request (ASCII request) at all. No Data, No Flags, No Reply, No Nothing. It needs to issue a reply to every request. DavidK 09/25/01 Wave_viewer often crashes during shutdown. This is a harmless bug that only occurs when you close wave_viewer. Sorry for the annoyance/inconvenience. Adsend: Automatic restarts of adsend (using the "restartMe" line in the descriptor file) can cause an NT system to hang. Therefore, you should never use the autorestart feature with adsend, but you should bring down the entire Earthworm system if adsend needs to be restarted. LDD 5/31/2000 Comments added to adsend.desc, but leave this warning here! libsrc/utils/site.c: The strings used for station, channel and network are required to be fixed length with trailing spaces added to short names. If the strings given to site_index do not have these trailing blanks, SCN names will not match. This is not documented anywhere. PNL 10/15/00 sm_ew2ora: There is a bug in sm_ew2ora that involves having multipe time intervals for components and channels. If a strong motion message containing information for a channel that the DB has never seen before is loaded into the DB, and then later another message for the same channel with an earlier timestamp is loaded, the load of the second message will fail, due to problems with overlapping time intervals, and the call that sm_ew2ora uses to create those time intervals. This problem only affects stations that were not previously loaded via one of the station loader programs stalist*2ora, and only when receiving SM data that is timestamped with a time that is prior to the original time for that channel. The bug lies in the logic of ewdb_api_PutSMMessage(), and not in the underlying code. Davidk 05/25/01 A change was made to ewdb_api_PutSMMessage() that dramatically affects sm_ew2ora. Please see the note about that function. Davidk 2001/07/26 KNOWS DEFICIENCIES: ******************* ringtocoax: In Windows NT, the time resolution of sleep_ew() is about 16 msec (one clock tick). On Solaris, the resolution is about 10 msec. This is a problem for ringtocoax, since packet delays need to be set to a few milliseconds. statmgr: A space is needed between "tsec:" and the value. If it isn't there, things fail. Artifcat of the kom routines. (Alex) threads functions: The KillThread function on WindowsNT and Solaris terminate the thread without ensuring that no mutexes are held. If a thread holds a mutex when it dies, no other thread can get that mutex. PNL 1/12/2000 carlsubtrig: The system time must be set to GMT and ew_nt.cmd must have TZ=GMT for carlsubtrig to work. Comments in ew_nt.cmd done 5/25/00. Barbara localmag: needs GMT set on the system ew2seisvole: on NT, exits with horrible crash when system is stopped. NUMBER OF RINGS LIMITED ON SOLARIS: Under Solaris 2.6 (and probably other versions as well), the maximum number of shared memory segments is six. This means that on an out-of-the-box machine you can only configure six rings. If you try to configure more than that, you will see a cryptic message from tport_create about too many open files. The fix to this problem is to add the following lines to the /etc/system file, and then reboot the system. set shmsys:shminfo_shmmax = 4294967295 set shmsys:shminfo_shmmin = 1 set shmsys:shminfo_shmmni = 100 set shmsys:shminfo_shmseg = 20 set semsys:seminfo_semmns = 200 set semsys:seminfo_semmni = 70 This allows for 20 rings. Lucky Vidmar (7/6/2000) startstop_solaris: There MAY be a problem with the signal that startstop sends to modules during the shutdown sequence. The shutdown sequence is started (after typing "quit" to startstop or running "pau") by startstop placing a terminate message on all transport rings. Modules should see this message and start their own shutdown. After a configurable delay, startstop checks to see that all modules have exitted. Any that are still running are sent a signal to terminate them. Currently that signal is SIG_TERM. But since wave_serverV has a handler for SIG_TERM, wave_serverV sees that as essentially the same as a terminate message. So if wave_serverV is having problems completing its shutdown, SIG_TERM won't do anything. The result is that startstop may give up and exit, leaving wave_serverV running. If that happens, the operator will have to terminate wave_serverV by doing "kill -9 ". That may leave shared memory and semaphores stranded in the kernel: run the command "ipcs -a" to see. If necessary, the stranded shared memory and semaphores may be cleaned up with the ipcrc command; must be run as root; see the man page. This problem only exists on Solaris/Unix, not on WindowsNT. PNL, 10/4/2000 libsrc/utils/kom.c: fix comment in k_open() The comment above k_open() says that only one file can be open at a time. Yet the Kbuf array has slots for MAXBUF (currently 4) open files. Does this work, or is the comment to be taken at it's word? PNL 10/15/00 libsrc/utils/logit.c: logit_init() requires a module_id number, which it uses to construct the log file name. This is not helpful, since the module_id number is not meaningful to people. Worse, it requires that the config file be read and earthworm.d lookups be completed before logit calls can be made. Thus errors in the config file can only be reported to stderr or stdout instead of being saved in a file. PNL 11/29/00 TRACEBUF messages. The definition of `endtime' of the TRACEBUF message is not documented. Some programmers are taking it as the "expected start time of the next TRACEBUF packet (if the sample interval is uniform.)" The more accepted practice is that `endtime' is the time of the last sample of the current TRACEBUF packet; that is, one sample interval less than the expected start time of the next TRACEBUF messsage. Using this last definition, if a TRACEBUF packet has exactly one sample, then its starttime and endtime are the identical. Clearly this distinction needs to be documented. The file waveform_format (in the /home/earthworm/DOC directory) gives no specifics about start or end times. PNL 1/24/01
Date last modified: April 16, 2002