Geospatial Imaging - Data Management and Archiving

The definition of a “large” storage system has changed over the years. Only ten years ago, a 1-terabyte storage system was considered quite large. Today, an individual home may have a terabyte of storage under its roof considering computer systems, iPods, and DVRs. Businesses face the same challenge with increasing capacity needs and more difficult data management challenges. These issues are furthers magnified for geospatial imaging organizations like GeoEye (see GeoEye Success Story) whose data needs are daunting, even by today’s standards. The good news is that storage technologies such as RAID and Tape systems have grown in size and sophistication to meet today’s data storage needs and data management solutions like StorNext have provided automated mechanisms to match business needs with technology.

Storing the Earth without breaking the bank

Overview

There is nothing more important for understanding our environment than understanding the Earth we live on. Today’s technological breakthroughs have allowed us to study and capture more information about the earth than ever before. There are a multitude of methods for capturing this information including seismological information, weather models, and a variety of remote sensing techniques. While all of these earth observation techniques provide unique data about the earth, they all have one thing in common: they consume vast amounts of data. This data’s value varies over long periods of time with the most current data universally delivering the most value as it relates to current events. As time progresses, the data becomes less current and, hence, less valuable. However, when a customer is interested in tracking changes over time, suddenly the older information is just as valuable as the new data. The combination of old and new data is even more valuable as the two pieces are merged to collectively answer customer’s questions. This paper focuses on the management challenges in archiving and repurposing data by utilizing examples in geospatial imaging environments, most notably with aerial and satellite remote sensing. The challenges, and suggested solutions, however, apply to any environment generating and analyzing large quantities of data.

Business Needs

More and more corporations are “data driven.” A company’s success is often driven by how it manages its data. This is particularly true for businesses whose primary product is data based such as companies capturing, manipulating, and creating unique assets from geospatial imaging information. These organizations specialize in creating airborne or space-based remote sensing platforms that capture images of the Earth using sensors including infrared, electro-optical (visible), and radar.

As with any of today’s technology-driven businesses, the drive to improve the technological base, stay ahead of the competition, and satisfy customer requirements is unrelenting. For these geospatial companies, this translates to higher resolutions and larger scan areas resulting in significant increases in file sizes. And with improvements in communication technologies, airborne and space-based platforms transfer more imagery data, at higher data rates to ground stations for processing, analysis, and delivery to customers.

This sheer volume of data presents significant challenges to geospatial imagery businesses. These businesses need to maintain focus on their core business of capturing and providing value added imagery data to their customers but storage and data management issues are consuming a larger portion of information technology resources and budgets. These organizations are challenged to adequately design and economically operate long-term state of the art storage systems.

Solutions

Today’s storage devices only provide point solutions to specific portions of large data repository issues. For example, high-end Fibre Channel based RAID solutions provide excellent performance profiles to meet demanding throughput needs of geospatial acquisition and processing systems. Serial ATA (SATA) based RAID systems provide large capacity needs for longer-term storage. Tape storage provides enormous capacity without power consumption. Each of these storage technologies also has unique downsides to their utilization for certain data types. Utilization of Fibre Channel RAIDs for all data types is cost prohibitive. SATA-based RAID systems do not have the performance or reliability to stand up to high processing loads. Tape technology does not facilitate transparent, random access for data processing.

Architecting a solution leveraging the aspects of each device’s capabilities into a seamless, scalable, managed data repository is the focus of the remainder of this white paper. Quantum’s StorNext software, which provides high speed data sharing and long term archiving, is detailed to help readers identify how to architect a final solution.

Download the complete Geospatial Imaging - Data Management and Archiving White Paper