white paper netapp e series performance benchmarks using ... · 3 netapp e-series performance...
TRANSCRIPT
White Paper
NetApp E-Series Performance Benchmarks Using Landmark SeisSpace/ProMAX Melinda McDade, Senior Performance Benchmark Engineer, NetApp
Steve Rovarino, HPS Rack Product Manager, NetApp
Larry Fink, SeisSpace/ProMAX Product Manager, Landmark Software and Services
Bryan Cote, Senior Product Manager, Terascala
August 2013 | WP-7170
2 NetApp E-Series Performance Benchmarks Using Landmark’s SeisSpace/ProMAX
TABLE OF CONTENTS
Introduction ................................................................................................................................................. 3
About Landmark Software and Services ................................................................................................................. 3
About Terascala ....................................................................................................................................................... 4
About NetApp .......................................................................................................................................................... 4
NetApp High-Performance Storage Rack ................................................................................................. 5
HPS Rack Architectural Overview ............................................................................................................ 6
TeraOS ......................................................................................................................................................... 7
Benchmark Configuration .......................................................................................................................... 8
Benchmark Description and Results ........................................................................................................ 8
Write Performance ................................................................................................................................................... 9
Read Performance ................................................................................................................................................... 9
Benchmark Conclusions .......................................................................................................................... 10
LIST OF FIGURES Figure 1) Landmark SeisSpace/ProMAX. ....................................................................................................................... 4 Figure 2) NetApp E5460 storage system. ....................................................................................................................... 5 Figure 3) HPS rack designed for flexibility, performance, and scalability. ...................................................................... 6 Figure 4) Components of TeraOS. ................................................................................................................................. 7 Figure 5) Landmark Denver lab HPS rack configuration. ............................................................................................... 8 Figure 6) GeoUser write results. ..................................................................................................................................... 9 Figure 7) GeoUser read results. ................................................................................................................................... 10
LIST OF TABLES Table 1) SeisSpace/ProMAX version 5000.8.2.4 benchmark results. ............................................................................ 9
3 NetApp E-Series Performance Benchmarks Using Landmark’s SeisSpace/ProMAX
Introduction Like so many industries today, the oil and gas sector has its big data challenges. Data gathered and used for oil and gas exploration, specifically prestack seismic data, has passed the petabyte threshold and is well on its way to exabytes. Moore’s law of increasing channel count1 predicts that the channels in a single seismic survey will grow to well over a million by 2030. This expansion is largely due to the introduction of new shooting and recording technologies. Wireless seismic recording, simultaneous source acquisition, full azimuth (FAZ), wide azimuth (WAZ), rich azimuth (RAZ), and multiazimuth (MAZ) are acquisition technologies and geometries being applied in the field today that are leading us into the exascale environment.
The processing and modeling of this massive amount of data require new ideas around IT infrastructure, especially with regard to data storage systems and the requirement for new levels of application and processing throughput.
NetApp, with its E-Series storage solutions for high-performance computing, has joined forces with Landmark Software and Services and Terascala to design a high-throughput seismic processing storage architecture. Called the Landmark Accelerated Processing Architecture, it provides a new, higher level of throughput, performance, and supportability for the seismic processing industry. The Accelerated Processing Architecture uses a tighter integration with Landmark’s industry leading SeisSpace/ProMAX software, the NetApp® high-performance storage (HPS) rack solution based on E-Series storage systems, and the Terascala operating system (TeraOS).
This white paper describes the joint architecture and initial performance results that place NetApp, Landmark, and Terascala at the leading edge of seismic processing.
About Landmark Software and Services Landmark is the premier provider of software and technology services for the upstream oil and gas industry. Under the Landmark umbrella, SeisSpace/ProMAX is the industry-leading comprehensive seismic processing system for large-volume land and marine 2D and 3D prestack and poststack data.
The system combines intuitive use with effective analysis tools, geophysical algorithms, and an optimized parallel infrastructure. With SeisSpace/ProMAX software, you can derive greater value from your investment in seismic data, increase processing productivity, reduce project cycle time, and better understand your subsurface and reservoir targets. SeisSpace performs equally well for in-house processing teams and for contractors who are trying to keep up with the growing number of surveys in their backlog, enabling them to deliver large volumes of data, quickly and accurately.
With productivity and cycle times in mind, Landmark researchers in the Denver R&D facility continually strive to obtain the maximum performance from their systems. Landmark’s infrastructure support staff worked with the engineers from NetApp to identify and resolve a number of I/O performance bottlenecks in the Denver data center infrastructure. Performance and scalability of NetApp E-Series systems were tested against large prestack JavaSeis datasets. The JavaSeis format is designed to take advantage of large compute clusters with the ability to read and write very large volumes in parallel. The new E-Series hardware proved particularly well suited to the demands placed on the system from large JavaSeis volumes. Landmark researchers were able to read and write JavaSeis data at a rate of 3.2GB/sec or better to the NetApp HPS rack..
1 SEG, The Leading Edge, Monk 2006.
4 NetApp E-Series Performance Benchmarks Using Landmark’s SeisSpace/ProMAX
Figure 1) Landmark SeisSpace/ProMAX.
About Terascala Terascala greatly accelerates the time to insight for enterprise R&D by eliminating I/O bottlenecks associated with simulation, analysis, and modeling applications. By accelerating the time to insight, customers have a major advantage in bringing new products and/or knowledge to market.
The Terascala operating system is the enabling software, which is distributed across system components to create a single high-performance storage appliance. TeraOS orchestrates the dependencies between the software and the hardware, simplifying the appliance management, monitoring, and tuning so that users can focus on their applications, not the system.
TeraOS was developed from Terascala’s deep expertise in Lustre parallel file system management, application performance tuning, and system integration. TeraOS delivers high availability and stability, fast deployment measured in days instead of weeks, application performance tuning, simplified troubleshooting, as well as historical data for reporting and capacity planning. The included reporting capability allows for optimizing ever-changing workflows in real time.
About NetApp NetApp creates innovative storage systems and software that help customers around the world store, manage, protect, and retain one of their most precious corporate assets: their data. NetApp is recognized throughout the industry for continually pushing the limits of today’s technology so that our customers never have to choose between saving money and acquiring the capabilities they need to be successful.
NetApp always finds ways to enable our customers to do things they couldn’t do before at speeds they never thought possible. We partner with industry leaders to create the most efficient and cost-effective solutions optimized for their IT needs and to deliver and support them worldwide.
Figure 1) Landmark SeisSpace/ProMAX.
5 NetApp E-Series Performance Benchmarks Using Landmark’s SeisSpace/ProMAX
The NetApp E5460 is a high-performance storage system that meets an organization’s demanding performance and capacity requirements without sacrificing simplicity and efficiency. Designed to meet wide-ranging requirements, its balanced performance is equally adept at supporting high-performance file systems, bandwidth-intensive streaming applications, and transaction-intensive workloads. The E5460 multiple-drive shelf options enable custom configurations that can be tailored for any environment.
With over 20 years of NetApp storage development experience behind it, the E5460 is based on a field-proven architecture designed to provide the highest reliability and 99.999% availability. Its redundant components, automated path failover, and online administration help keep organizations productive 24/365. The advanced protection features and extensive diagnostic capabilities of the E5460 consistently achieve the highest levels of data integrity.
NetApp High-Performance Storage Rack NetApp’s HPS rack offers flexibility and capacity to meet the current and future capabilities of SeisSpace/ProMAX users. This solution provides a turnkey, open source, high-performance parallel file system solution with advanced management and monitoring capabilities. It is a fully supported, easy-to-deploy, and easy-to-use solution designed for complex and demanding workflows. The prepackaged and preconfigured storage solution scales as storage requirements grow, resulting in faster time to results with lower TCO.
The NetApp HPS rack appliance is an integrated system based on the NetApp E-Series storage platform, servers based on Intel®, Lustre high-performance file system, and TeraOS. The appliance includes all MDS servers, OSS servers, NFS/CIFS gateways (if needed), storage controllers, drives, and related hardware and software needed to meet the needs of the users.
HPS rack can be deployed in one day and is fully integrated and tested before it arrives at the customer’s site. Pretesting and validation reduce deployment time and eliminate the majority of installation failures and incompatibilities that commonly occur during on-site installation and integration. Lustre software, the Linux® operating system, and all management tools are preloaded. TeraOS provides ease of management and extensive analytics of the environment.
Figure 2) NetApp E5460 storage system.
6 NetApp E-Series Performance Benchmarks Using Landmark’s SeisSpace/ProMAX
Figure 3) HPS rack designed for flexibility, performance, and scalability.
HPS Rack Architectural Overview The HPS rack includes the following Lustre components: • Terascala management server (MGS)
− Provides system monitoring and management • Active-passive metadata server (MDS)
− Manages directories, files, permissions, access • NetApp E2624 metadata storage target (MDT)
− Provides storage for Lustre metadata information • Active-active object storage servers (OSSs)
− Provide I/O services, manage user data − Present storage from object storage target
• NetApp E5460 object storage targets (OSTs) − Provides the user data storage to the attached OSS
• Gigabit Ethernet switch for the management network • Lustre version 1.8.8 (version 2.1.5 will be available in fourth quarter of 2013) • TeraOS
7 NetApp E-Series Performance Benchmarks Using Landmark’s SeisSpace/ProMAX
TeraOS
The HPS rack integrates TeraOS to deliver extensive management and analytics to HPC environments. The HPS management system builds on the proven Lustre parallel file system technology to bring rock-solid reliability. Administrators can go beyond a simple "break/fix" scenario to gain insight into hardware failures, application issues such as resource contention, and file system tuning opportunities.
TeraOS collects key performance indicators for all major hardware and software components within the HPS rack, including MDSs/OSSs, storage arrays, interconnects, operating system, and Lustre file system. It also monitors client/application access patterns, giving a 360-degree view of the health and performance of the storage appliance:
• A high-level dashboard provides an easy-to-understand summary with targeted drill-down capability for detailed analysis and problem resolution.
• Individual sensor values are correlated and combined to detect abnormal conditions that are communicated to the user in the form of problem-level alerts.
• The base scanning interval for sensors is 5 seconds; however, this can easily be changed.
TeraOS offers the following:
• Ease of management. Reduce management complexity of parallel file system appliances. • Simplicity. Designed for engineers and researchers with no extensive parallel file system experience
to easily manage and maintain their environment.
Figure 4) Components of TeraOS.
8 NetApp E-Series Performance Benchmarks Using Landmark’s SeisSpace/ProMAX
• Depth of control. Navigate from top-level views to specific component details and individual LUN usage within three clicks of the mouse.
• Responsiveness. Automated notifications assist support staff while enhancing response time. • Optimization. Complete visibility into application service levels enabled by a simple user interface
that leverages real-time data collection. • Planning. Leverages data collected over time to enable accurate planning and optimization of overall
system performance.
Benchmark Configuration Figure 5) Landmark Denver lab HPS rack configuration.
Benchmark Description and Results Table 1 presents the results of the SeisSpace/ProMAX application throughput while running the GeoUsers (www.geo-users.org) I/O performance benchmark. These tests consist of creating, reading, and processing a 3TB JavaSeis dataset (960 extents) that closely mimics the Society of Exploration Geophysicists Advanced Modeling (SEAM) Phase I marine seismic data (http://www.seg.org/resources/research/seam/seamphasei).
Note: The I/O throughput reported is based on the application I/O with respect to the total application runtime. Therefore the reported I/O bandwidth is somewhat less than the actual aggregate I/O reported by the E5460 controllers while each test was running.
9 NetApp E-Series Performance Benchmarks Using Landmark’s SeisSpace/ProMAX
Table 1) SeisSpace/ProMAX version 5000.8.2.4 benchmark results.
HPS Rack Configuration
Number of SeisSpace Hosts
Write Measured MB/sec
Simple Read Measured MB/sec
Write Throughput MB/sec
Simple Read Throughput MB/sec
96 3,259 3,456 3,016 2,821
Each SeisSpace Java® process (ssexec) consists of approximately 30 lightweight threads, so the best I/O performance is achieved by running one process per client host.
Write Performance With regard to write performance, a single process per 96 client nodes on the same switch achieves 3016MB/sec SeisSpace/ProMAX write throughput performance. This is approximately 50% of the 6055MB/sec peak of the six 10GbE uplinks and includes the job initialization, network logging, MPI, and performance monitoring of SeisSpace overhead.
Figure 6) GeoUser write results.
Read Performance The read workflow applied AGC, deconvolution, and band pass filtering on the input data automatically while reading the seismic traces. The read throughput performance is affected by this additional computational overhead, which was not seen in the write test scenario. This additional processing has a direct impact on the overall throughput capabilities of the system.
10 NetApp E-Series Performance Benchmarks Using Landmark’s SeisSpace/ProMAX
Figure 7) GeoUser read results.
Benchmark Conclusions The benchmark described in this white paper was done in coordination with Landmark Software and Services, NetApp, and Terascala. The HPS rack component of Landmark’s Accelerated Processing Architecture delivered application throughput results of 3016MB/sec for writes and 2821MB/sec for reads using NetApp E5460 storage systems, a Landmark 130-node development compute cluster, and the SeisSpace/ProMAX application. Had we been able to overcome the nonoptimal network configuration at the Landmark Denver Lab, we believe the HPS rack would have delivered even better performance using the Landmark applications.
Beyond the pure performance aspects of this joint solution, it is important to consider other attributes that affect the total cost of ownership. As seismic storage requirements grow, system efficiency, scalability, and supportability become critical factors. The performance per density of the Accelerated Processing Architecture allows maximum throughput in a small footprint, requiring less power and cooling, and using the NetApp Global Support structure means supportability. So as your seismic processing throughput and density requirements increase, the Accelerated Processing Architecture scales with you.
© 2013 NetApp, Inc. All rights reserved. No portions of this document may be reproduced without prior written consent of NetApp, Inc. Specifications are subject to change without notice. NetApp, the NetApp logo, and Go further, faster are trademarks or registered trademarks of NetApp, Inc. Linux is a registered trademark of Linus Torvalds. Intel is a registered trademark of Intel Corporation. Java is a registered trademark of Oracle Corporation. All other brands or products are trademarks or registered trademarks of their respective holders and should be treated as such. WP-7170-0813
NetApp provides no representations or warranties regarding the accuracy, reliability, or serviceability of any information or recommendations provided in this publication, or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS, and the use of this information or the implementation of any recommendations or techniques herein is a customer’s responsibility and depends on the customer’s ability to evaluate and integrate them into the customer’s operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.