Table of Contents Previous Chapter 1 The Project

1 The Project

1.1 Background

1.1.1 National Weather Service Modernization

For several years, the National Weather Service (NWS) has been engaged in activities directed toward modernizing and restructuring its operations. The activities include as major components the development of a new radar system (NEXRAD; the individual hardware units are known as WSR-88D), a new automated surface observing system (ASOS), and a new communications and forecaster workstation system, the Advanced Weather Interactive Processing System (AWIPS).

In order to make more effective use of the talents of its staff of professional meteorologists, the NWS plans to reorganize its operations. Currently, 52 Weather Service Forecast Offices (WSFOs) perform the bulk of the forecasting functions for statewide areas, and 180 Weather Service Offices (WSOs) and other small offices provide local adaptive forecasts. Both WSFOs and WSOs issue severe weather warnings, though many WSOs operate on a less-than-24-hour schedule and must be backed up by their parent WSFO at night.

In the restructured Weather Service, there will be 115 Weather Forecast Offices (WFOs), roughly collocated with WSR-88Ds. AWIPS will provide the communications and forecast support functions for these offices.

1.1.2 Acquisition of AWIPS

An AWIPS Requirements Task Team (ARTT) was formed in the early 1980s, comprising representatives of NWS administrative, development, and field offices, plus what is now the Forecast Systems Laboratory (FSL). The work of the ARTT was used to refine and validate AWIPS requirements prepared by the NWS Office of Meteorology. These requirements form the basis of the functional requirements included in the AWIPS Request for Proposal (RFP) and the AWIPS Development Phase contract.

The AWIPS requirements include several hydrometeorological techniques. The functions of these techniques range from decoding, analyzing, and displaying observations to formatting official forecasts. Detailed specifications for these techniques have been prepared by the NWS Techniques Development Laboratory (TDL), working closely with other NOAA elements, in the form of Technique Specification Packages (TSPs).

In preparation for the development and deployment of AWIPS, the NWS has asked FSL to participate in several risk reduction activities. The primary activity has been developing and testing a series of forecaster workstations at the Denver and Norman WSFOs. Currently, systems known as DARE (Denver AWIPS-90 Risk Reduction and Requirements Evaluation) in Denver and Pre-AWIPS in Norman are being used by the WSFO staffs. These systems have provided valuable insight into modernized operations; experience gained in Denver over the past five years has been used to refine the specifications that are being used by the AWIPS contractor.

The AWIPS Extended Definition Phase, wherein two competing contractors developed detailed plans and proposals for the system and forecaster workstations, has just been completed.

The AWIPS Development Phase contract was awarded to Planning and Research Corporation (PRC) on 29 December 1992. Contractor software is to be delivered in several "builds," with the first due approximately one and one-half years after contract award; the second, known as the First Article Capability (FAC), to be completed approximately two years after contract award; and the third, known as the Initial Deployment Baseline (IDB), scheduled for delivery three years after contract award.(1)

Of the many techniques included in the AWIPS requirements, several will not be developed by the AWIPS contractor but will be furnished by the government. The government will furnish software to the contractors in two Pre-Planned Product Improvements (P3I), which will take place after the IDB delivery. Two of the major software developments by the government are a grid-based forecast system and a full suite of product generators. Collectively, these constitute the AWIPS Forecast Preparation System.(2) The software and documentation described in this document will be delivered to the contractor as part of the second P3I.

1.2 Objective

The specific objective of this project is to develop a set of techniques that will allow the forecaster to efficiently issue a full suite of products. Improving the efficiency of the forecast generation process will permit forecasters to focus more on hydrometeorology and allow for more timely updates of forecast products to improve the quality of the forecasts. Further, the use of product generators will improve product consistency.

1.3 NWS Requirements and Assumptions

In Appendix A of the AWIPS System Requirements Specification, Volume I (part of the AWIPS-90 Request for Proposal, dated 22 August 1990), the grid-based forecast system is outlined as follows:

The WFO will maintain an up-to-date official set of digital forecasts, usually in gridded form, over this [750 kilometers by 750 kilometers] area. From this set of forecasts, routine forecast products shall be automatically formatted for release after forecaster review and, if necessary, modification. (RFP p. SRSI-A-10).

1.3.1 Requirements

The AWIPS IDB will include many of the basic hydrometeorological techniques required to support forecast operations at WFOs. These include improved observation decoders, improved quality control techniques, and improved analysis and display techniques. Graphical techniques will be used to enter and edit Watch/Warning/Advisory information and to format these products automatically. Also included will be interactive techniques to enter information for selected forecasts as well as techniques to format selected public, agricultural, and hydrologic observations and forecasts such as today's weather roundup. Techniques to assist in the preparation of additional products, including aviation, marine, and fire weather programs, will be deferred until the two P3I upgrades of AWIPS.

At the heart of these upgrades is a forecast database, containing gridded and geographically located weather elements that support forecast preparation, quality control, monitoring, and verification. Forecasters will maintain this database, starting with first-guess fields "rolled over" from previous forecasts or based on one or more centrally- or locally-run numerical or statistical models, and drawing on their experience and training. This project will develop prototype techniques to refine the concept and integrate the resulting system into the AWIPS workstation.

The NWS requires techniques to assist the forecaster in preparing most, if not all, routine forecasts as well as watches, warnings, and advisories. These techniques must facilitate the maintenance of a forecast database which supports all program areas. The database must have appropriate time resolution to allow the creation of all forecast and data products, it must contain quantitative and qualitative variables needed by the products, and it must support related programs such as forecast monitoring and verification.

The interactive techniques used by the forecaster must allow specification and control of the weather elements necessary to generate products and perform other forecast functions. The number of elements should be kept low to minimize the amount of information the forecaster must enter. Information should have to be entered or edited only once, not multiple times for each program area or forecast function. In addition to the general forecast, the techniques must permit the entry of local forecast information to define the effects of terrain and bodies of water on the forecasts.

AWIPS will automatically generate as many forecast and warning products as is reasonable and partially format those that will still require considerable manual entry. The system must be capable of producing text, graphics, and gridded forecast products, as well as broadcast-ready text products for NOAA Weather Radio (NWR). Taken as a whole, these techniques must streamline the forecast process when compared with today's operations, and they should improve the consistency of the forecasts. They should enhance interoffice coordination and optimize the use of central and local guidance. The system of preparing forecasts and generating products should allow for the rapid changes that are sure to take place in the future as additional high-resolution guidance information becomes available.

1.3.2 Basic Assumptions

Several basic but critical assumptions have been made in this plan. If any of these assumptions is incorrect or cannot be realized, the implementation schedule and/or the breadth of the final project will be affected.

1.3.2.1 Timely Delivery of AWIPS Equipment

To meet the goal of transferring forecast preparation software to the AWIPS contractor for integration into the system by the second P3I, both FSL and TDL must receive standard-configuration WFO AWIPS systems (Government Development Platforms, or GDPs) soon after contract award. The basic COTS (commercial off-the-shelf) hardware and software platform are scheduled to be received 12 weeks after contract award,(3) with FAC software and documentation delivered at the second scheduled software build (approximately two years after contract award).

1.3.2.2 AWIPS Contractor Assistance

During the Prototype Stage of FSL's development (see Chapter 9 for schedule information), we will be developing generic code using UNIX, C++, OI (Object Interface, a C++ Graphical User Interface library), and XGL (Sun's version of GL, an industry-standard graphics system incorporating parts of and compatible with X) on a network of Sun computers. Once AWIPS hardware and System Software are available, we will build our first integrated system on AWIPS.

Due to the short time allowed to build the first integrated operational prototype on AWIPS, we must have direct contact with the AWIPS contractor. Our schedule includes approximately one year between receipt of AWIPS System Software and the commencement of WFO risk-reduction activities. During this period, we will need to "come up to speed" on the internals of AWIPS and integrate perhaps 250,000 lines of code into AWIPS. FSL's Advanced Development Facility (ADF) group has been involved with AWIPS proposal evaluations, and will attend design reviews and study the internals of AWIPS. They will serve as consultants in this work. Direct contact with the AWIPS contractor will also be essential.

The assistance needed takes several forms. We would like to make a formal presentation and demonstration of our plans to the AWIPS contractor so they can gain an appreciation of the magnitude of the task. Once they are familiar with our goals, they can offer guidance on the preferred method of integrating this system into AWIPS. Once we start integrating code, the contractor could serve as a consultant. Ideally, we would work daily with the contractor and have one or two members of the contractor team on site in Boulder.

1.3.2.3 FSL/TDL Will Port to AWIPS

FSL will port the visualization and editing software into AWIPS for the Risk Reduction Stage tests. Our software will be visually integrated into AWIPS at this time but may not actually use all of the AWIPS System Software (i.e., it may use some parallel implementations of system functions). "Visually integrated" means that the user will not perceive any separation between our software and the rest of AWIPS, although the code will not be truly integrated. Actual integration will be performed by the AWIPS contractor, since modifications to the AWIPS System Software could be required.

TDL will port product generators to the AWIPS platform (GDP) to make them available during the Risk Reduction Stage tests. Some will be supplied to the AWIPS contractor for the IDB; others will be prepared for the first P3I.

1.3.2.4 Risk-Reduction WFOs are Unique Sites

We assume that the Risk-Reduction WFOs (Denver and a to-be-determined marine office) will be unique sites in that their AWIPS will not be under strict configuration management by the AWIPS contractor.

We will need to load our prototype software at these sites for risk reduction testing. While it is imperative to keep the contractor informed of configuration changes, we must not be restricted from making such changes. This issue will be coordinated with the AWIPS Program Office.

1.3.2.5 Static AWIPS Architecture

AFPS will eventually become an integrated part of AWIPS. It will utilize many of the sophisticated graphic processing features of the AWIPS hardware. As a result, we will be making design decisions based on information obtained from the AWIPS contractor. We assume that the basic hardware and software configuration of AWIPS, e.g., operating system, windowing system, graphical bit plane allocation, will remain static during our development.

1.3.2.6 Forecasters Will Work with Weather Elements

Forecasters will be asked to forecast general and service-specific weather elements, not all product parameters, once this system is deployed. General weather elements are a few observable items such as temperature, winds, and clouds, and service-specific elements include such fields as wave heights and dew intensity. In contrast, product-specific items such as heat index/wind chill, hours of sunshine, and maximum temperature may not need to be edited by the forecaster. To minimize forecaster workload, the number of edited weather elements must be kept to a minimum.

1.3.2.7 Techniques Not Designed for RFCs and NMC

We are developing a software package to meet the needs of the WFO forecaster to produce routine forecast products. It is not intended to satisfy all interactive drawing requirements of River Forecast Centers (RFCs) or the National Meteorological Center (NMC).(4)

Although there may be technical common ground among various government groups writing AWIPS code, AFPS is being designed to support WFO operations, and thus the list of weather elements, support data (e.g., background maps), and grid/time resolutions will be built from a WFO perspective, and by itself will not support NMC or RFC operations. We expect that a GDP technical forum comprising NMC, the NWS Office of Hydrology (OH), the NWS Office of Systems Development (OSD), and FSL will be formed by OSD's Advanced Development and Demonstration Laboratory to discuss GDP development issues. This group would meet periodically to discuss new ideas and problems.

NMC is preparing a document which will present plans for development within NMC to provide forecast products required to support an operational AFPS at WFOs. Specifically, NMC must be able to provide a set of gridded products to WFOs sufficient to initialize the AFPS forecast database.(5)

1.3.2.8 Common Product Formats

The NWS issues many different types of products. Many of those products today, such as agricultural and public forecasts, contain similar types of information; however, their formats differ. If common formats can be specified for various products, then the required number of discrete product generators will be reduced. For example, if the zone forecast (or its successor) uses the periods today, tonight, and tomorrow, and the agricultural forecast uses the same format, common code can be developed for both.

It is assumed that the NWS will examine the products in the AWIPS environment and determine which products can be formatted similarly. If the products continue to have different formats, then more product generator code will have to be developed.

1.3.2.9 Forecast Database Will be Used by Related Programs

As noted above, forecasters will edit general and service-specific weather elements. Algorithms will sample the database and create additional product parameters. Forecast monitoring and quality control programs will refer to this database.

Eventually, verification should also be based on the forecast database. For now, the verification program will remain as is, using such products as the CCF (coded cities forecast). The National Verification Committee is responsible for planning any changes in this program.

1.3.2.10 Forecasters are Allowed to Edit Text Products

NWS forecasters will be allowed to edit the text produced by product generators using standard AWIPS capabilities. This principle is enunciated here because it has the potential to compromise the effectiveness of the forecast database concept. In Chapter 10, we propose solutions to the problem of uncontrolled text editing.

1.3.3 Integration with AWIPS

FSL procured development hardware and began investigative work before the AWIPS platform became available. Although this forecast preparation system eventually will become a fully integrated part of the AWIPS System Software, we will take a step-wise integration approach. The software will first be installed as an AWIPS application program while we start risk reduction exercises and are gaining knowledge of AWIPS. Later, the AWIPS contractor will integrate it into the AWIPS System Software.

We must be cognizant of the integration process from the outset of the project. The effort required to integrate existing software must be weighed against the effort required to "start from scratch" and develop new routines that would serve the same functions.

A final issue to be considered in integration is forecaster training. Depending on AWIPS deployment schedules, AFPS could be released to all 115 WFOs simultaneously. The forecasters will have become familiar with AWIPS by that time; however, this new system will introduce novel capabilities that will change the way forecasts are prepared. Thus, substantial training requirements should be anticipated.

1.4 The Digital Forecast System

The AWIPS grid-based forecast system requirement is intended to streamline forecast generation. By itself, the system is not intended to improve the accuracy of forecasts, though it may incidentally contribute to that goal by freeing the forecaster from routine chores and allowing more time to study the weather.

Simplified, forecasting comprises three steps (which overlap in practice): examination of data, models, numerical and statistical forecasts, analogs, climatology, yesterday's successes and failures, etc.; mental formulation of a forecast; and (text) composition. Although this new system will not change this process, it will reduce the mechanical and repetitive composition phase, giving the forecaster relatively more time to apply his or her knowledge and expertise to forecasting.

The central component of this system is the forecast database of weather elements, describing the present and future state of the atmosphere in the forecast area. Values in this database are initialized and/or updated with observations and forecast values derived from national and local models, including forecast grids produced manually at NMC. The forecaster modifies and maintains the forecast database to create the best representation of the future state of the weather in the forecast area according to his or her experience and judgement, based on the complete set of weather and model information available through the AWIPS workstation. Interactive graphic tools allow the forecaster to maintain and modify the database. Automatic consistency checkers examine the internal consistency of the values in the database, and monitors compare the forecasts represented by the database with adjacent WFO forecasts for boundary consistency, with new observations and guidance, and with the current forecast. Significant deviations are brought to the forecaster's attention for consideration and action where needed.

All required text products are generated when appropriate (by forecaster request or on a schedule) by automatic text generators which compose English-language or coded text products from the information in the database. The forecaster has the option of editing the text products, but will generally do so only to improve clarity or description of weather and not to revise the forecast; doing otherwise would compromise the validity of verification based on the forecast database.(6) One goal of the NWS modernization is to provide a set of products that will routinely require no editing by the forecaster.

The details of how this system will be built will be understood only after investigative development and risk reduction evaluations. Given their experience, FSL and TDL are well qualified to do this work.

1.4.1 Project Scope

An essential factor for successful completion of this project is an understanding of the scope of the project from the beginning.

In the modernized National Weather Service, the forecaster will have three techniques to assist in creating forecast and warning products (Figure 1 on page 8):

This project encompasses development of the Graphical Forecast Editor and interactions with/development of the AWIPS user interface, Warning Generation, Product Preparation, and the forecast database.

Creating forecast products supported by the GFE involves three steps: creating first-guess forecast weather elements (from, e.g., a previous forecast, climatology, numerical and statistical guidance), editing these elements, and generating products. The forecaster will import selected guidance and/or previous forecasts into a graphical worksheet and then will use interactive editing tools to modify the depictions until the desired forecast solution is created. This solution will be stored in the database as the official forecast. Products will be generated on command of the user.

The concept of manipulating a forecast database and automatically producing formatted products is not new. The Interactive Computer Worded Forecast System (ICWF) has been under development at TDL since 1985. Using Model Output Statistics (MOS) and Local AWIPS MOS Program (LAMP) output as a first guess, ICWF produces a limited set of forecast products. Depending on the outcome of current operational evaluations, it will become part of the AWIPS IDB or first P3I. AFPS will build on the ICWF concepts and provide even more capabilities to the forecaster.

The primary difference between the ICWF and AFPS, other than differences in scope, is the method of editing forecasts. With the ICWF, the forecaster manipulates a matrix of zone forecast numbers and symbols to specify forecast elements; with AFPS, the forecaster will graphically manipulate a picture to specify weather elements.

1.4.2 Overview of the System

The following five components constitute the AWIPS Forecast Preparation System:
Figure 2 describes the system data flow. The process begins by retrieving guidance and observations from the AWIPS database(7) and generating general weather elements. These elements may be derived from model output or observations, or simply copied from a previous forecast. The forecaster will manipulate these fields using graphical editing tools. Interpolator tools will fill in snapshots that were never initialized. Service-specific elements are computed from the general elements and other information in the AWIPS database. Any changes made to these elements cannot be reflected back to the general elements. At the forecaster's request, the weather elements are checked for inconsistencies, and the forecaster may resolve them. Next, samplers extract point, area, and route forecasts from the database and convert them into specific product parameters. Finally, a product generator uses the product parameters to generate a forecast in text form. The forecaster reviews the product with the AWIPS word processor and makes any necessary wording changes. The product is then stored and disseminated to users.

1.4.3 Definition of Tasks

The goal of this effort is to produce a system via which a forecaster can produce a full suite of forecast products by visualizing and modifying a representation of atmospheric weather elements. To achieve this goal, the following tasks have been defined.

1.4.3.1 Phase I Tasks

The work described in this plan will focus on the development of a basic system, to be deployed two to three years after the AWIPS IDB, and will include development of the five components listed above.

In the initial work, the database will be in essence a two-dimensional, ground-based representation of the atmosphere.(8) The limitation to two dimensions is imposed to reduce the scope of the near-term work.

A full suite of product generators will be developed during Phase I. These generators will initially work with the ICWF database, but later will be modified to work with the weather elements in the database.

1.4.3.2 Phase II Tasks

Work necessary to make the forecast system fully functional includes the following:
These longer-term tasks are further addressed in Chapter 10.

1.4.4 Related Projects

A few government organizations have been involved with projects that are similar to AFPS. We are examining these projects in order to take advantage of others' expertise in this work. Other NWS groups also have experience with modern computer networks and computer graphics, notably OH, which has been working on RFC-related systems, and the NWS Alaska Region, where forecaster-programmers have developed a number of UNIX/X applications to support forecast operations.

We have examined the experience of the NWS with the Aviation Route Forecast project of the early 1980s. Two current projects are the ICWF developed by TDL and the Forecast Production Assistant (FPA) developed by the Canadian Atmospheric Environment Service (AES). Since the latter two are in many ways similar to AFPS, we describe some details of them below.

1.4.4.1 ICWF Project

In the early 1970s, TDL began development of an automated computer worded forecast system (CWF). It initially produced city forecasts from MOS guidance. In the early 1980s, the program was expanded to include zone forecasts and terminal forecasts. The forecaster was presented with the end product, a text forecast, for editing.

The ICWF program began in 1985 to develop a text generator for AFOS. It presented the forecaster with a zone-based basic weather matrix. The forecaster could interactively view and modify the matrix before the text forecast was generated. This version of the ICWF was limited to the forecast projections available from MOS guidance. A demonstration of ICWF began at several WSFOs in June 1986. WSFO Charleston still actively uses the ICWF, and much of its operation is built around the ICWF digital database.

The Forecast Entry and Formatting System (FEFS) program started in the summer of 1987. FEFS was based on a grid of digital weather elements, with higher temporal and spatial resolutions than the zone-based ICWF. It provided both zone-based and grid-based viewing and modification tools. The grids were initialized from LAMP, MOS, and Perfect Prog techniques. Before it could be implemented, the FEFS project was suspended in 1989 by the NWS in favor of porting ICWF to the Norman, Oklahoma, Pre-AWIPS demonstration system. What was learned in FEFS is being considered in the design of AFPS.

In 1990 and 1991, the ICWF program was enhanced to take advantage of many things that TDL learned from the FEFS development. The current version of ICWF uses a new set of weather elements that makes it easier for a forecaster to depict the desired weather. The Pre-AWIPS version of ICWF has become a part of the Norman risk reduction exercises. If the evaluation is favorable, ICWF will be ported to AWIPS as part of the IDB or the first P3I.

Many of the concepts developed for ICWF will be incorporated into AFPS, such as point and area forecast extraction from a grid and the station model plot depiction.

1.4.4.2 FPA Project

The FPA is the Canadians' approach to the automatic creation of forecasts from a digital weather element database. It allows forecasters to draw a series of weather depiction forecasts and to produce a set of computer-generated forecasts. When the FPA project is complete, it will produce all public, aviation, marine, and forestry products. It will also produce graphical products for specialized users.

The FPA concept began in 1985 when a student of computational linguistics took a series of hourly temperatures and created a statement of the temperature trends and abnormal situations. In 1988, the FPA project was funded and development began.

The first goal was to produce a marine forecast from weather depictions of five sensible weather elements. The elements used were surface temperature, surface pressure, clouds and weather areas, wind deviation from geostrophic, and wave height. AES conducted several risk reduction exercises in Halifax, Gander, and Toronto. Version 1.0 of the FPA became operational in the spring of 1991 at Gander, Halifax, and the Arctic Weather Centre. One interesting requirement of the product generators for the FPA is that they produce forecasts in both English and French.

FPA development continues. It was determined that a Forecast Bulletin Editor was required to accurately create the forecast since the weather depiction charts could not always be made to properly represent the weather. The Bulletin Editor provides a time series depiction of the forecast at a point, allowing the forecaster to fine-tune the forecast after the weather depiction charts have been drawn. The Canadians do not permit editing of the text product.

AES plans to expand the FPA to include public forecasts in 1993, and forestry, agriculture, and aviation forecasts by 1996. Local models will help create these products starting in 1996.

AFPS will draw on many of the concepts developed for the FPA. Its weather depiction capabilities and interactive graphical editing tools provide a working example of how a forecast production system might operate. However, there are three important conceptual differences between the FPA and AFPS. The set of weather elements with which the forecaster works will differ, the value of computational linguistics for text generation compared to the ICWF approach is unclear, and whether the FPA graphical techniques are applicable to mesoscale forecasting is unknown.


Footnotes

(1)
An overview of the AWIPS development schedule and its relationship to the work described in this document is found in Section 9.3.
(2)
Since this system will be an integral part of AWIPS, it need not be singled out by name; however, for convenience we will refer to it throughout this document as .
(3)
FSL's GDP equipment was delivered on 29 March 1993; TDL's (NWS HQ) on 5 April.
(4)
NMC is in the process of reorganizing and modernizing its service operations. NMC will become the National Institutes for Environmental Prediction (NIEP) and the national centers will be restructured to become the Aviation Weather Institute, the Marine Prediction Institute, the Tropical Weather Institute, the Storm Prediction Institute, the Climate Prediction Institute, and the Weather Prediction Institute.
(5)
The forecasts provided by NMC will generally be for projections beyond six hours. Local information (e.g., observations and short-range forecasts) will have to be integrated with these fields.
(6)
This is a future concept only. As noted in Section 1.3.2.9, verification will remain text-forecast-based until NWS policy directs otherwise.
(7)
Although it appears in Figure2 that there are several databases, all are part of the AWIPS database.
(8)
Requirements for 3-d data (in particular, for aviation terminal forecasts) can be met in our 2-d representation by including height information in the data (e.g., 50SCT).
 
Table of Contents Next Chapter