1.  INTRODUCTION

     National Weather Service (NWS) forecasters at the Denver, Colorado, Weather Service Forecast Office (WSFO) have used AWIPS-90 like (Advanced Weather Interactive Processing System) workstations (DARE I and DARE II) for several years.  Since 1991, fore-casters at the WSFO in Norman, Oklahoma, have also used similar workstations (Pre-AWIPS).  These workstations have provided the forecasters with many advanced datasets and capabilities that will be available nationwide later this decade.  Evaluating how forecasters use the workstations will provide the NWS with im-portant risk reduction information before a system is deployed nationally.
     This paper summarizes the 1994 warm-season workstation product usage patterns from the Denver and Norman WSFOs.  It is a companion paper to a cool- season comparison study that was conducted during winter 1992-1993 (Roberts and Kucera 1993).   The analyses indicate which workstation products (such as model, radar, surface, and satellite) are most commonly used by forecasters. Particular emphasis is placed on how forecasters are using advanced products such as Doppler radar and high-resolution numerical weather prediction model output, as well as the Oklahoma State Mesonet (Crawford et al. 1992) used by the Norman staff,  and new NEXRAD WSR-88D radar used by Denver fore- casters.  Differences attributable to product availability, staffing, and weather are discussed.

2.  DATA COLLECTION AND SAMPLE

    All actions made by the forecasters on the work- stations' graphics devices are recorded. (Product calls, retrievals, loads, and requests are used synonymously in this report to describe when a graphic or image was brought to the screen by a user action.)  The workstation-generated log files record the time the products were displayed, along with the type of product, product scale, map backgrounds, and color tables.  Additional infor- mation such as product overlays and looping are also recorded.  Log files were collected and evaluated from Norman and Denver during their respective warm, severe weather seasons, namely March through June 1994 at Norman, and April through July 1994 at Denver.

3.  DIFFERENCES BETWEEN DENVER AND NORMAN

     The Denver and Norman WSFOs are both involved in NWS risk reduction activities.  This has allowed NWS to introduce and test new workstations, datasets, tech- nologies, and paradigms in a small number of offices before they are implemented throughout the rest of the forecast offices.  However, the experimental and developmental nature of these activities contributes to differences between the offices, which may account for some of the differences found in product usage.  Major differences and similarities are listed below.
    Workstation Configuration - The Norman office was configured with four full-function workstations (two animating graphics screens and one alphanumeric screen each) and one text workstation (one alphanumeric screen).  The long-term, short-term and warning forecasters were each assigned to a workstation.  Hydrometeorological Technicians (HMTs) used the fourth full-function workstation.  The service hydrologist used one screen of the warning workstation.  The Denver office had three full-function workstations, which were used inter- changeably for warnings and short- and long-term forecasting during the spring and summer of 1994, and two text workstations.  One of the full-function workstations was shared by the HMT.
     Product Availability -  Both offices received satellite products from NESDIS (National Environmental Satellite Data and Information Service), and gridded numerical model output and products from the National Meteorological Center (NMC).  Denver also received high-resolution model grids and products produced from the Mesoscale Analysis and Prediction System (MAPS) (Benjamin 1989) and the Local Analysis and Prediction System (LAPS) (McGinley 1989), which are run locally at the Forecast Systems Laboratory in Boulder.  Both offices used MAPS surface products (Miller and Benjamin 1992), which were available only on the regional scale at Norman.  Denver's Doppler radar data were supplied from the WSR-88D (NEXRAD) radar which was accepted in May 1993 and commissioned in July, 1994.  Norman received Doppler data from the Twin Lakes and Frederick WSR-88Ds.
     Finally, Norman forecasters had access to the Oklahoma State Mesonet, which has been providing 5- minute averages of surface parameters from 111 surface observation sites deployed across the state of Oklahoma since late 1993 (Shellberg et al. 1993).  The Norman WSFO receives updates every 15 minutes on the graphics workstations and can also access mesonet plots on DOS-based personal computers.  Because usage logs are not generated for the PCs at Norman, only the Mesonet use on the Pre-AWIPS workstations will be discussed in this paper.  Denver forecasters continued to use the high- resolution surface mesonet plots, which have been available for over 10 years.  Previous studies have shown that mesonet products are one of the most frequently requested products in Denver (Heideman et al. 1989, Roberts et al. 1992, and Roberts and Kucera 1993).  In addition, Norman forecasters routinely accessed gridded model data via PC-GRIDDS, which is a DOS-based program that allows interactive access to NMC gridpoint data on a PC.  These graphics products were not included in the Pre-AWIPS usage logs.
     Staffing - Both offices typically had two forecasters and one meteorological technician on duty around the clock. And both sites had an additional forecaster available in the event of severe weather.  The duties, geographic areas of responsibility, and forecasts produced by forecasters were somewhat different in each office.  Denver forecasters had statewide responsibility for county zone forecasts, as well as aviation forecasts, storm watches and statements, and local warning and forecast products for northeast and north-central Colorado, including the Denver metropolitan area.  Norman forecasters were responsible for county zone and aviation forecasts, as well as watches and warnings for the western two-thirds of Oklahoma and eight counties in north-central Texas.
     Weather -  Preliminary storm data compiled at both offices indicated that it was a fairly active severe weather season over Oklahoma and Colorado.  The severe weather in Oklahoma included large hail (up to softball size on several occasions), heavy downpours, and damaging winds.  At least 14 F0 and F1 tornadoes were reported, with eight occurring in April.  Record-breaking heat and below-average precipitation characterized the warm season in much of Colorado, with most of the severe storms occurring along an abiding dryline in extreme eastern Colorado.  Although tornadic activity was uncharacteristically sparse, there were about five reported F0 tornadoes.  A persistent ridge over the inter-mountain region kept the flow of subtropical moisture from reaching Colorado and thunderstorm development was unusually suppressed.  Denver experienced its hottest June on record.

4.  PRODUCT USAGE PATTERNS

     A total of 129,456 product loads were logged at Denver during the four months of April - July 1994.  Norman recorded 101,750 product loads from March to June 1994.  It is interesting to note that more products were loaded at Denver, which has one less workstation than Norman.  Possible reasons for this disparity may include training, product availability, staffing, and weather as mentioned above.

4.1  Commonly Used Products

     Tables 1a and 1b list the 16 most-requested products during the 1994 warm season at Denver and Norman, respectively.  These products represent approximately the top 30% of the total number of product requests for each location.  The national scale infrared (IR) satellite image was ranked highest at Denver with 3084 calls, while this same product at Norman was seventh, with 2002 calls. The Denver mesonet plot, ranked second, was accessed 2954 times on the subWFO-scale and 1443 times on the WFO scale.  From previous studies by Heideman et al. (1989) and Steiner et al. (1992), the subWFO mesonet plot was usually ranked highest during the warm season months, while the national IR satellite was ranked highest in the cool season months, especially to observe wide- spread clouds and precipitation.  One possible reason for the change in the order of these two products may have been the unusually hot and dry warm season experienced in Colorado.  Denver forecasters may have been viewing larger scale satellite imagery to observe the onset of the seasonal monsoon.  Among the top products,  Satellite data (IR, WV, and visible) were heavily used by fore- casters at both sites on the national and regional scales.  The regional scale visible image had similar use at both Denver and Norman, with frequencies of 2574 and 2524, respectively.
     As expected, NEXRAD radar products were well-represented in the top products at Norman.  The Twin Lakes 0.5° Reflectivity on the local (state) scale was loaded most often (4291 times) by Norman forecasters, as shown in Table 1b.  The Twin Lakes 0.5° Z/SRM (Reflectivity/Storm Relative Motion, a radar scan which allows forecasters to load radar reflectivity and velocity with one keystroke), was loaded 3656 times on the WFO scale.  The corresponding Frederick radar products were also among the most-frequently used products at Norman, and were loaded 2983 and 2596 times, respectively.  The Vertically Integrated Liquid (VIL) products on the WFO scale of both the Twin Lakes and Frederick radars had frequencies of 1612 and 1347, respectively.  One higher elevation angle image, the Twin Lakes 1.5° Z/SRM on the WFO scale, was included in the top 16 products at Norman and had a frequency of 1452 calls.  Also, the national- and regional-scale Radar Summary was used by Norman forecasters 1431 times.  The extensive use of these WSR-88D radar products may reflect the confidence that Norman forecasters have with these data, as well as the relatively active weather that occurred during this four month study.
     At Denver, the Front Range Z/V 0.5° radar image on the WFO scale was the most frequently requested radar product and was loaded 2748 times.  The same product on the subWFO scale was loaded 1651 times over the four months.  We expected more use of the Front Range WSR-88D radar products during its first warm severe season in operation, but the unusually dry weather in Colorado may have limited this.
      The regional-scale Surface Aviation Observation (SAO) plot, which was likely overlaid on the regional scale visible and IR satellite imagery (also in the top-16), was noted among the 16 most frequently used products in Norman and Denver, with frequencies of 3514 and 1973, respectively.  Lightning data also appeared in the top products, although the graphics were somewhat different.  Denver forecasters used the 30-minute accumulated lightning plot, on the regional, and local scales, 2552 times.  On the local scale, Norman forecasters used the lightning plot 1613 times over the four months.

      Several vertical-scale products were among the top 16 most-used products at Denver.  The Platteville profiler was called 1394 times and the Denver Skew-T, 1649 times.  No vertical-scale products appeared on the list of most-requested products from Norman; however, the regional-scale profiler plot (a plan view of the entire profiler network at constant height) at 850 mb was used 1807 times by Norman forecasters.
      In Table 1a, two MAPS surface products appear among Denver's most-requested products; the 3-hour surface pressure change and the surface wind analysis, which were both on the regional scale.  No MAPS surface products were noted in Norman's top 16 products, although several were among the top 50% of Norman's warm season product list.
      The new Oklahoma mesonet has quickly become a popular workstation product at Norman.  From March to June 1994, it was used 1409 times, or about 11 times per day on the local or WFO scales.  Keep in mind that the forecasters also view the Oklahoma mesonet data on PCs, which are not recorded in the Pre-AWIPS usage logs.  Therefore, the above frequency underestimates the actual use of these data.  On benign weather days, Norman forecasters call-up the Mesonet on the PC an estimated 45 to 60 times per day, and up to 150 times per day when precipitation or frontal boundaries are in the Mesonet domain (W. Ruff, Risk Reduction Meteorologist, Norman WSFO, personal communication).

      Family graphics products allow forecasters to access up to eight fields of data, usually from the same numerical model, with a single mouse click.  Two families, the Aviation (AVN) and the Regional Analysis and Forecasting System (RAFS) are in Table 1a.  The AVN Family was loaded 1763 times and the RAFS, 1483 times by Denver forecasters.  The lack of severe weather in Colorado may explain the emphasis Denver forecasters made of the larger-scale model data.  Family graphics did not appear on Norman's list of most frequently used products.  The focus was on smaller (mesoscale) products during the warm season, and the larger scale model data were not used as often.

4.2  Products by Category Type

     Product categories group similar types of workstation products.  These categories include Surface, Satellite, Radar, Vertical, Upper Air, Model, Grid-to-Graph, Applications, and Other/Miscellaneous.  Table 2 shows a distribution of all requested products by category for both Denver and Norman during the 1994 warm season.  The percentage of each product category is provided, along with the daily mean, which was calculated over 122 days, and the standard deviation.

     The first five categories include all observational datasets, followed by Model, Grid-to-Graph, Applications, and miscellaneous products.  A discussion of the distrib- ution of product requests within each category follows:

     Surface-  Products in the Surface category accounted for 22% of all product loads at Denver, with a daily average of 233 products per day (sd=57).  Thirteen percent of product loads at Norman were in the Surface category with a daily mean of 106 loads per day (sd=42).  At Denver, 39% were MAPS surface products, 21% were LAPS surface products, and 16% were mesonet plots.    MAPS surface analyses made up 34% of the Surface category at Norman, and the regional-scale SAO plot, 27%.  An additional 11% came from the Oklahoma mesonet plot.  Recall that Norman had MAPS products only on the regional scale and had no LAPS model products.

     Satellite-  Denver used more satellite imagery during this study than Norman.  The Satellite category represented 12% of total products at Denver (mean=126, sd=22) and 9% at Norman (mean=76, sd=22).  Infrared (IR) imagery on all scales had the greatest use at both stations: 43% at Norman and 44% at Denver.  Visible imagery, available during daylight hours and on the various scales, were used 40% at Norman and 35% at Denver.  Water vapor (WV) imagery at Norman and combined WV/IR imagery at Denver were the least used.

     Radar-  As expected, Norman, with access to two WSR-88D radars, showed over twice as much use in radar data as Denver.  Most (29%) of all product requests at Norman were in the Radar category, with a daily mean of 241 calls per day (sd=287).  This notably large stan- dard deviation reflects the large variability of daily use of Radar products.  In contrast, Radar products at Denver made up 12% of all products and were used an average of 129 times per day (sd=80).  As mentioned earlier, the anomalously dry spring in Colorado may have contributed to this difference.  The packed Z/SRM radar scans represented most (46%) of all Radar products at Norman, followed by Reflectivity with 29%, NEXRAD and Hydrology algorithms and products (e.g., the one- and three-hour rainfall accumulation, Weak Echo Region images and overlays, and reflecivity cross sections) with 17%, and packed Z/V with 1%.  At Denver, the packed Z/V scans were used most (46%), with the algorithms making up 32%, the Z/SRM, 4%, and reflectivity, 3%.  The Norman forecasters have had more experience with the Z/SRM imagery, which may explain its notably greater use than the  Z/V scans.  The Radar Summary and Legends, which are often overlaid together (Kucera and Roberts 1994), represented 13% of the Radar category at Denver and 5% at Norman.

     Vertical-  Vertical products accounted for 11% (mean=121, sd=30) and 3% (mean=20, sd=15) of all product loads at Denver and Norman, respectively.  The Vertical category distributions for both Denver and Norman were made up of the traditional upper-air soundings plotted on the Skew-T charts, and the time- height plots of wind profiler data.  At Norman, 38% of the Vertical products were profiler data, and the remaining 61% were Skew-T plots.  Skew-T plots represented 78% of the Vertical product frequency at Denver, and profiler plots made up the remaining 21%.  As expected, the profiler sites located closest to the WSFOs were accessed most often (e.g., Purcell, Oklahoma, had 259 calls and Platteville, Colorado had 1394 calls).

     Upper Air-  These products are plots of upper-air data on constant altitude or pressure surfaces.  They accounted for a small percentage of products used at Denver (3%, with a daily mean=28 and sd=11) and Norman (4%, with a daily mean=35 and sd=15).  Norman requested profiler plan-view plots more often than Denver (86% versus 39% of the Upper-Air products).  This was likely because Norman is located in the middle of the profiler network, which allows optimal observation of up- and down-wind weather features.

     Models-  Numerical model products are retrievable in different ways.  As discussed earlier, selecting model "families" accesses several fields and forecast times of a particular model run with a single mouse click.  Additionally, the AFOS graphics option allows the user to select specific models, levels, and fields through a matrix menu.  Another method of obtaining model data is by generating graphics from gridded data, which is not included in the Model category and will be discussed in the next section.  At Norman, 25% of the total product loads at were in the Model category, and averaged  207 loads per day (sd=84).  Denver recorded 8% within the Model category, with a daily mean of 88 loads per day (sd=22).  Fifty-eight percent of Denver's Model use was attributed to family graphics, whereas only 3% of the Norman Model category was family graphics.  One disadvantage of the family graphics, as expressed by several Norman forecasters, is the inability to redraw the graphics to a finer resolution after zooming in on a geographical area.  Instead, Norman forecasters selected most of the Model category products via the matrix menus, which typically involves more "layers" of menu selections.

     Grid-to-Graph-  A third way to display model data and derived model fields (e.g., isentropic data) is through a workstation feature called Grid-to-Graph, whereby graphical products are generated locally from NMC gridded data, contoured, and sent to the graphic/image display at the time the request is made.  Selecting Grid-to-Graph products usually takes more time because one must go through multiple menu levels.  Additionally,  time is required to generate the product onto the screen, from 5 to 20 seconds on average.  The great benefit of the Grid-to-Graph capability is that it provides access to a large array of graphics not available through NMC-generated graphics.  As shown in Table 2, 21% of all products requested at Denver were loaded via Grid-to-Graph, which translates to about 226 loads per day (sd=55).  In contrast, 10% of all Norman product requests were in the Grid-to-Graph category, with a daily mean of 80 calls per day (sd=50).  MAPS Grid-to-Graph products (only available in Denver) accounted for over one-third of the Grid-to-Graph products calls in Denver.  Grid-to-Graph products from all three NMC models were requested at Norman and Denver.  It should be noted that Norman forecasters routinely use gridded model data on the PC via PC-GRIDDS, as discussed in Section 3, which are not recorded in the Pre-AWIPS usage logs.

     Interestingly, the data in this study, as in the Cool Season report (Roberts and Kucera, 1993) revealed that when the number of Model product loads and Grid-to-Graph product loads from both Denver and Norman were totalled, their sums were rather close: 38,302 Model and Grid-to-Graph products at Denver, and 34,974 at Norman.  While the total model use is similar, the display methods and fields used were considerably different.  Denver forecasters appear to obtain a "quick look" at the forecast from the families and then rely on Grid-to-Graph for more detail.  At Norman, it seems forecasters use the model matrices for both a cursory look and for detail.

5.  EFFECTS OF SEVERE WEATHER ON PRODUCT USE

     The effects of severe weather on workstation product usage were also examined.  We patterned this analysis after previous studies by Walker (1990) and Steiner et al. (1992), which surveyed product usage during warm season severe weather at Denver.  Kucera and Roberts (1994) compared product use during cool season severe weather at both Norman and Denver.

5.1  Definitions and Summary

     Severe weather days were defined as those days on which at least one severe weather warning was issued.  At Norman, 37 out of the 122 days studied were severe weather days, while Denver had 30.  The last rows of Tables 3a and 3b summarize the overall statistics from Norman and Denver, differentiating between the nonsevere and severe days.  The columns labeled "prod count" contain the number of different products within each Product category that were loaded by the forecasters; these values do not reflect how often each different product was loaded.  The average number of product calls per day at Norman was 701 for nonsevere days and 1139 on severe weather days.  Denver forecasters averaged 1029 calls per nonsevere day and 1159 calls per severe weather day.

     It is interesting to note, from the product counts, that even though there were fewer different products viewed by forecasters on severe weather days versus nonsevere (862 and 1052, respectively at Norman, and 1373 and 1901, respectively at Denver), the average number of calls per day was higher on severe days.  Forecasters appear to be looking more often at fewer different products on days with severe weather.  This trend was consistent with the previous severe weather studies mentioned above.

5.2  Product use by Product Category

     Tables 3a and 3b compare the relative use of workstation products by Product categories for nonsevere and severe weather days at Denver and Norman, respectively.  The percentages of each Product category are included, as well as the mean number of loads per day.  The last column in Tables 3a and 3b shows the change in the mean relative to severe weather days.  The percentages of each product category at Denver were fairly uniform from the nonsevere to severe days.  The largest difference was in the Radar category, which increased from 10% and a mean of 106 loads per day (sd=66) on nonsevere days, to 17% on severe days, with a mean of 200 loads per day (sd=79).  This was not surprising since radar imagery was the primary tool for detecting most severe convective weather.  Also, the WARNGEN application requires radar imagery to generate severe weather warnings.  Norman had an even greater change in the Radar category from nonsevere to severe days.  The Radar category accounted for 16% of the products on nonsevere days, with a mean of 115 calls per day (sd=136).  On severe days, the Radar category made up 47% of the total calls, with a mean of 530 calls per days (sd=332).  In fact, there was one severe day when Norman forecasters used 1261 radar products.  Having access to two WSR-88D's had a great impact on these statistics, as well as the growing radar experience of the Norman staff.

    Daily use of the Surface category increased significantly during severe weather at both sites.  At Denver, Surface products increased by 37 loads per day, and at Norman by 31 loads per day.  Note that while Surface products were used more on severe days, the number of different Surface products dropped on severe days, from 116 to 94 at Denver and 61 to 51 at Norman.  This follows the trend that on days with severe weather, forecasters load more frequently a fewer number of different workstation products.

     The Other/Miscellaneous category (which contains lightning and precipitation data) showed a substantial increase in use on severe days.  At Denver, the daily mean of Other category products increased by 23 calls per day, and at Norman, by 20 calls per day.  The number of different products decreased from 96 to 67 at Denver, and 78 to 71 at Norman.

      The Model and Grid-to-Graph categories showed a decrease in use on days with severe weather, which seemed logical, since model products are probably not as useful during severe weather operations.  The Vertical category also showed a decrease in use on severe days.

6.  CONCLUSIONS

      We have briefly discussed many of the similarities and differences in product usage between Denver and Norman during the 1994 warm season.  This study has shown that advanced products were used in both locations, including Doppler radar products and the
vertical and plan views of profiler data.  Norman took particular advantage of the latter because of its location near the center of the profiler network.  Denver and Norman routinely used high-resolution MAPS surface products.  Denver also exploited MAPS upper-air products and LAPS products.  The Oklahoma mesonet products were frequently utilized by the forecasters at Norman.  It was shown that during severe weather,  forecasters appear to be looking more often at fewer different products.  More detailed analyses of product usage will be topics of future evaluations.
 

7.  ACKNOWLEDGMENTS

      The authors would like to thank D. Salisbury, who provided substantial technical and software support.  We also acknowledge the Denver and Norman WSFO staff for their valuable input.  Many fellow FSL staff members helped explain workstation options and products, including J. Ramer and R. Lipschutz.  We also thank E. Thaler, C. Lusk and J. Fullerton for their review of this manuscript.

8.  REFERENCES

Benjamin, Stanley G., 1989: An isentropic meso-scale analysis system and its sensitivity to aircraft and surface observations, Mon. Wea. Rev., 117, 1586- 1603.

Crawford, K.C., F.V. Brock, R.L. Elliott, G.W. Cuperus, S.J. Stadler, H.L. Johnson and C.A. Doswell III, 1992: The Oklahoma Mesonet - A 21st Century Project.  Preprints from the 8th International Conference on Interactive Information and Processing Systems for Meteorology, Oceanography, and Hydrology, Amer. Meteor. Soc., 27-33.

Heideman, K.F., D.C. Walker, and J.A. Flueck, 1989.  DARE-I evaluation: An overview. NOAA Technical Report ERL 426-FSL2, NOAA Environmental Research Laboratories, FSL, Boulder, CO, 29pp.

Kucera, Patrice C. and W.F. Roberts, 1994. Cool season product usage patterns from the DARE workstations at the Denver and Norman WSFOs.  NOAA Technical Report (In press), NOAA Environmental Research Laboratories, Boulder, CO,.

McGinley, John A., 1989: The local analysis and prediction system.  Preprints, 12th Conf. on Weather Analysis and Forecasting, Monterey, Amer. Meteor. Soc., 15-19.

Miller, Patricia, A. and S.G. Benjamin, 1992: A system for the hourly assimilation of surface observations in mountainous and flat terrain, Mon. Wea. Rev., 120, 2342-2359.

Roberts, William F., E.J. Steiner, and C.M. Lusk, 1992: Product usage patterns from the DARE workstations at the Denver WSFO, Preprints, Eighth International Conference on Interactive Information and Processing Systems for Meteorology, Oceanography, and Hydrology, 5-10  Jan 1992, Atlanta, GA, Amer. Metor. Soc., 247-252.

Roberts, William F. and P.C. Kucera, 1993: Cool season product usage patterns from the DARE workstations at the Denver and Norman WSFOs.  Preprints, 13th Conf. on Weather Analysis and Forecasting, 2-6 Aug 1993, Vienna, VA, Amer. Meteor. Soc., 522-525.

Shellberg, D.J., K.C. Crawford, and L.J. Ruthi, 1993: Nowcasting capabilities using the Oklahoma mesonet.   Preprints, 13th Conf. on Weather Analysis and Forecasting, 2-6 Aug 1993, Vienna, VA, Amer. Meteor. Soc., 29-32.

Steiner, Ellen J., W.F. Roberts, and C.M. Lusk, 1992.  Use of DARE-II workstation products and capabilities in the summer of 1990.  NOAA Tech. Rep. ERL FSL-2, NOAA Environmental Research Laboratories, Boulder, CO, 46pp.

Thaler, Eric R. and P. C. Kucera, 1993:  The Denver Mesoscale Forecasting Initiative: What we have learned.  Preprints, 13th Conf. on Weather Analysis and Forecasting, 2-6 Aug 1993, Vienna, VA, Amer. Meteor. Soc., 185-188.

Walker, D.C., 1990.  DARE-I evaluation: forecasters' assessment and use of the DARE-I system during the 1988 warm season.  NOAA Tech. Rep. ERL 441-FSL 4, NOAA Environmental Research Laboratories, Boulder, CO, 55pp.