GOES Data on the Internet - A Tutorial

URL Dennis.F.Chesters@nasa.gov
GOES Project Scientist
Goddard Space Flight Center
National Aeronautics and Space Administration

Abstract

The GOES-8 and GOES-9 satellites carry an Imager that regularly scans large rectangular regions-of-interest several times per hour using 5 spectral bands. The Imager's data stream is broadcast and used by NOAA to make weather forecasts. Anyone can set up a ground station for a cost between $25k and $250k, and make a pool of digital images from GOES's "firehose" data stream.

NOAA is not in the business of providing real-time data directly to the public. So, other government agencies, educational institutions, and commercial organizations collect and distribute GOES images. With the recent opening of the information highway, GOES images are now routinely offered at several internet sites for scientific applications and public enjoyment.

This document describes the most popular sites and image formats available to the public at the end of 1995.

Table of contents

This document is part of a course on GOES satellite meteorology presented at the American Meteorology Society's (AMS) annual meeting at Atlanta, GA in January 1996.


GOES data formats

Raster image file formats consist of There are over 50 "standard" digital image formats and invented by workers in photography, printing, televison, movies, medicine, communications, and computers (Brown and Shepherd, 1995).

For GOES data, the most commonly used public-domain image formats on the internet are GIF, JPEG, HDF, TIFF and MPEG. The only proprietary format in wide-spread use for GOES imagery is the MCIDAS satellite data format.

Summary

GOES Digital Image Formats
FORMAT
--------
ATTRIBUTE
GIF JPEG MPEG TIFF HDF MCIDAS
Use screen print, TV TV movies print, screen
science
science science
Comm. Modem Modem Ethernet Ethernet Ethernet Tape, Net
Users Web Web Web Print
Computer
NASA NOAA
Calibrated no no no yes yes yes
Navigated no no no no yes yes
Multi-channel no no no yes yes no
Digital word 8-bit 2- to 24-bit 2- to 24-bit 1 to 32-bit 8,16,32-bit 8,16-bit
Color model LUT YCrCb YCrCb LUT, RGB
most others
LUT
RGB
none
Compression LZW
1.5:1
lossless
DCT
2:1 to 16:1
lossy
DCT w/dup.
2:1 to 64:1
lossy
LZW/JPEG/tile/...
1.5:1 to 64:1
loss(less/y)
RLE
1.5:1
lossless
none
1:1
lossless

Glossary with Notes

GIF - Graphics Interchange Format
In the 1980's, Compuserve developed GIF as an open standard for exchange of digital color pictures among personal computers. GIF was adopted as the standard image format for Web browsers.
JPEG - Joint Photographic and Engineering Group
In the early 1990's, computer graphics organizations agreed upon JPEG, which actually are a suite of 29 encoding options for making a highly compressed but lossy file format. JPEG optimizes the appearance of color photographs on a TV monitor by selectively truncating the high-frequency color components that are not easily transmitted on TV or noticed by the human eye. Today, inexpensive commercial software/hardware implementations provide only a simple "compression ratio" option. NOAA is planning to adopt JPEG compression for future high-speed GOES Imagers.
MPEG - Moving Picture Experts Group
In the mid-1990's, MPEG was devised to show movies on broadcast TV, including audio. MPEG imagery is based on JPEG, and it also uses only the difference between successive frames to update images, resulting in highly compressed but lossy movies. Commercial MPEG codecs are now under vigorous development.
TIFF - Tagged Image File Format
In the 1980's, TIFF format originated for transporting 8-bit digital greyscale imagery within the print industry, and is currently managed by the Aldus Corp.. In the 1990's, TIFF was also adopted by the mini-computer and medical communities for storage and exchange of high-precision lossless digital images from modern "remote sensing" diagnostic systems. The TIFF standard is currently managed by Adobe Corp.. TIFF now encompasses almost every major raster-image encoding scheme (Ritter, 1995). Consequently, few TIFF readers can handle all possible options.
HDF - Heirarchical Data Format
The National Center for Supercomputing Applications (NCSA) at the University of Illinois (UI) invented HDF in the 1980's to deal with the problems of transporting multi-dimensional numerical model data along with image visualizations of its contents. In the 1990's, the HDF format was adopted and extended by NASA for distributing datasets from its Earth Observing System (EOS). HDF reader and writer libraries are still under development.
MCIDAS - Man-Computer Interactive Display and Analysis System
Int the 1980's, The Space Sciences Engineering Center (SSEC) at the University of Wisconsin (UW) invented MCIDAS format to deal specifically with calibation and earth-location information with NOAA satellite images. The most recent SSEC documentation of MCIDAS file format for satellite data, is the 1993 edition. SSEC is now working on the 1995 edition, which will accomodate the new GOES-I/M features. MCIDAS readers, writers, and datasaets must be purchased from SSEC.

RGB - Red, Green, Blue
computer words for each color on a monitor
CYMK - Cyan, Yellow, Magenta, blacK
computer words for each color in a printer
YCnCb - Luminance and 2-color Chrominance
computer words for brightness and color on broadcast TV
LUT - Look Up Table
256 color palette for 8-bit pixels
RLE - Run-Length Encoded
uses abbreviations for repeated bit patterns
LZW - Lempel-Ziv and Welsh
patented RLE compression scheme
DC - Direct Cosine Transform
"harmonic" analysis of NxN pixels, truncate finest details


GOES presentation formats

Digital Range
MEDIUM
--------
ATTRIBUTE
GOES Computer Monitor TV Print Eye
Bits 10 8, 32 8, 24 "4+" 8+ "5+"
Nominal Range
N:1 Levels
1024 256, 4E9 256, 16E6 16+ 256+ 32+
Practical Range 256 256 16 8 8 16

Q: How to map GOES data into the limited human range?
A: Reduce the dynamic range and highlight features to match the eye.

Brightness and contrast scaling
OUT = IN*A + B
Gamma correction, typically square-root
OUT = IN^0.5
Histogram Equalization
LUT(OUT) = HISTOGRAM(IN)
Colorized thresholds
COLOR(OUT) if > THRESHOLD(IN)
Multi-spectral color
RGB(OUT) = COMBINE(IN1, IN2, IN3)


GOES public data servers

Major internet servers of real-time GOES images

"Full time" GOES service is not quite possible on the internet because the satellite's broadcast data rate, 2 Mbits/second, is a little too fast for public cross-country transfers. Consequently, all the public GOES image file servers reduce either spatial resolution (mainly in the visible), regions covered (concentrating on sectors), or the update rate (only every few hours).
GOES image file servers
WHO
--------
HOW
NASA-ARC NASA-GSFC UW-SSEC
Where Ames Research Center
Palo Alto, CA
Goddard Space Flight Center
Greenbelt, MD
Space Science and Engineering Center
Madison, WI
URL ftp://explorer.arc.nasa.gov http://rsd.gsfc.nasa.gov
ftp://rsd.gsfc.nasa.gov
http://www.ssec.wisc.edu
gopher://gopher.ssec.wisc.edu
Satellites GOES-8
GOES-9
GMS-5
GOES-8
GOES-9

GOES-8
GOES-9

Region-of-interest,
update rate,
resolution


full Earth, all, 4 km
30 sectors, all, 1 km
7 regions, all, 4 km
full Earth, all, 16 km
Wisconsin sector, all, 1 km
CONUS region, 3 hr, 4 km
full Earth, daily, 32 km
Channels VIS, 0.6 um
IR2, 4 um
IR3, 6.7 um
IR4, 11 um
IR5, 12 um
VIS, 0.6 um
IR2, 4 um
IR3, 6.7 um
IR4, 11 um
IR5, 12 um
VIS, 0.6 um

IR3, 6.7 um
IR4, 11 um
Image formats HDF
GIF
JPEG
TIFF
JPEG
GIF
MPEG
(MCIDA$)
Age-off time 3 days 16 hours latest only
Radiometric
Calibration
counts
(convertable)
brightness temp.
albedo
none
(MCIDA$)
Earth Location
(Navigation)
scan
(convertable)
scan
(convertable)
none
(MCIDA$)
Overlay Maps none dots outlines
Re-gridding Lat-long, w/maps none round earth
Special Clients none NASA NOAA
Sounder on-line? no yes no
Main strength full earths high res. sectors (MCIDA$)



Notes on real-time, part-time, and off-line GOES servers


References

Key URLs for GOES stuff

NOAA weekly report on the GOES Project
http://www.osdpd.noaa.gov/EBB/post/goeswky.asc
NASA GOES Project Pages - status reports, image scrapbooks, and technical details
../goesproject.html
SSEC GOES pages and data products
http://oldthunder.ssec.wisc.edu/goes/goes.html
Realtime GOES and GMS full-earths at 4 km resolution - GIF, JPEG and HDF
ftp://explorer.arc.nasa.gov/pub/Weather
Realtime GOES sectors at full resolution for most of the Americas - TIFF and JPEG
ftp://rsd.gsfc.nasa.gov/pub/
Realtime screen-resolution globe and CONUS with map overlays - GIF
http://www.ssec.wisc.edu/data/index.html
Realtime screen-resolution globe and CONUS with map overlays - JPEG
http://www.cira.colostate.edu/earthsta/overview.htm
Other on-line sources of GOES stuff
../text/goesds.html


COPYRIGHT AND WARRANTY DISCLAIMER NOTICE

This information resource was created by one irresponsible employee of NASA.
It is in the public domain and permission is granted to use, duplicate, modify and redistribute it.
Please give credit for the satellite images to NASA-Goddard Space Flight Center.
NASA and its employees provide absolutely NO WARRANTY OF ANY KIND.



Mail to:
Dennis.F.Chesters@nasa.gov
Climate and Radiation Branch
Laboratory for Atmospheres
Earth Science Divison
Goddard Space Flight Center
National Aeronautics and Space Administration