芯片与x86芯片 - oracle.com · latency-aware kernel memory allocator (x86, sparc)...

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

RISC芯片与X86芯片选择面前，用户何去何从？

[email protected] 02/2015


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

3


议题

4

• 选型思路共享及探讨

• 虚拟化方案

• Solaris 操作系统

• SPARC 芯片

• SPARC 小型机产品线

• SPARC 路线图

• 总结


选型思路共享及探讨

5


RISC

• Reduced Instruction Set Computing – 精简指令集*，指令格式规格化简单化

– 高度优化指令集，指令执行周期短，

– 通用寄存器设计，简化并优化编译器设计

– Load/Store架构，单纯的内存寻址和访问，控制简单化

– 并行度更好，流水线效率更高

CISC

• Complex Instruction Set Computing – 复杂指令集*

– 通常没有也无法做到所有指令都高度优化，部分指令需要数十个周期

– 访问内存方式多样

– 汇编/机器码程序更小，节省汇编编程工作量，节约存储空间

6

讨论前的知识准备


RISC

• RISC就是小机

• RISC很贵

• 买RISC的越来越少

• 做RISC的也越来越少

CISC

• CISC就是X86

• X86很便宜

• 买X86的越来越多

• 做X86的越来越多

7

你可能听到这些……

所以结论是：我们的企业IT架构应该转向，从RISC平台向X86转移？


RISC

• 市场上常见的RISC芯片有

– SPARC T4/T5/M6, SPARC 64 X/X+, Power 7+/8, PowerPC, ARM

• SPARC服务器价格包含了

– 服务器硬件，Solaris操作系统，多种虚拟化功能与软件，软硬件支持

• 买RISC的少了

– 整体看减少明显，但企业用户还有大量使用

• 卖RISC的少了

– Oracle, IBM

CISC

• 市场上常见的CISC芯片有

– Intel Xeon/Core/Atom, AMD Opteron/Athlon

• X86服务器价格通常仅包含了

– 服务器硬件与硬件支持

• 买X86的多了

– 互联网用户是大头，企业用户的非重要外围应用也有不少用X86

• 卖X86的多了

– 芯片厂家仍然只有Intel与AMD

8

事实上是……


事实背后的事实有……

• 从 Pentium Pro (P6) 开始，Intel在CPU内部使用了IA-32-to-micro-op decoder

–指令集仍是CISC，底层则以RISC方式运行

–这是否说明RISC的性能优势与生俱来？！

• 2005年时Sun公司推出8核的SPARC T1芯片 –当时的Intel Xeon为2核，IBM Power 5 为2核，如今……

• 互联网行业的快速发展主导了X86的崛起 –互联网行业应用有其自身的特点——横向扩展，数据一致性要求低，不惧节点失效……

– Linux，开源软件……

9


硬件选型先谈软件

• 我们直接使用的是——软件选型的起点也应是——软件 –应用程序

–数据库

• 选型的思路应该是——自上而下

• 那么……问题来了…… 商用软件谁最强？商用数据库谁最强？商用操作系统谁最强？

• 哪家厂商可以做到软件硬件集成优化？

10


虚拟化方案

11

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 12

同时提高利用率与性能

多层次虚拟化

Server

HW Domain

Zone

LDOM

仅有Oracle提供如此丰富的多层次虚拟化功能

• 独一无二的创新

• 最小化开销

• 最大化利用

• 内置

• 免费


OS Virtualization Oracle Solaris Zones

• 上百个轻量级的虚拟机

• 动态、零开销

• Solaris 11 Zones, Solaris 10, 9, 8 Zones

• Kernel Zones

• Zone Clusters

• 最佳颗粒度

• 始自2005年

HW Virtualization Oracle VM Server for SPARC

Logical Domains

• 每服务器或硬件域最多128个LDOM

• 动态、低开销甚至零开销

• 操作系统隔离

• Domain Clusters

• 热迁移

• 始自2007年

13

多层次虚拟化


SPARC T5

• 16 cores, 128 threads

• 即使为每个虚拟机分配 2 线程，单 CPU 依然可以支撑 64 个虚拟机且不需要进行 CPU 超额分配

Xeon E5 v2

• 12 cores, 24 threads

• 在不做 CPU 超额分配时单 CPU 仅能支撑 24 个虚拟机

• 一旦使用 CPU 超额分配，则会发生大量的为虚拟机而进行的线程切换，这会带来 CPU 时钟片浪费，CPU Cache 有效率降低等问题，从而导致 CPU 开销增大

14

Oracle在芯片设计上的高瞻远瞩


Oracle T5-2

• 6 servers

• 192 cores

• 1,536 threads

X86 2-socket Servers

• 20 servers

• 480 cores

• 960 threads

15

投放 600 个虚拟机的对比


• 总投资包括 –硬件投资，另加3年服务

–服务器，相关机架与网络设施

–操作系统，虚拟化软件和管理软件 • X86配合操作系统与虚拟化软件为RHEL，

Red Hat Smart Mgmt，VMware vSphere Std.，VMware vCenter Server Standard

–外接存储

–净价为美国市场代理折后价

16

投放 600 个虚拟机的投资对比

Oracle T5-2 HP DL380p

Networking $246 $304

Storage $100 $208

System SW $0 $523

Servers $603 $447

$0

$200

$400

$600

$800

$1,000

$1,200

$1,400

$1,600

每 1 个虚拟机的投资

$533

$1,482

$949


打破软硬件界限的集成优化

Oracle SPARC虚拟化 vs. 典型X86虚拟化

RHEL

VMware

HP Server

Traditional Hypervisors Separate, isolated, slow

Oracle Solaris, Zones

Domains and Zones Efficient, robust, secure

Oracle VM Server

Oracle SPARC Server

深度整合


Solaris操作系统

18


If It Must Run, It’s On Solaris

Oracle Solaris 11

Best UNIXTM

for Oracle Deployments

Built for Cloud Infrastructures

#1 UNIXTM

for Enterprise Applications


传统的操作系统研发方式

Virtualization Network

Server

OS

早该有所改变了！


硬件、软件联合研发

Middleware

Database

Application

Virtualization&

Cloud Orchestration

SDN

Server

Platform

• 更强大的性能 • 更可靠的运行 • 更简单的使用 • 更优秀的效率 • 更丰富的价值


冰山一角

Solaris 针对 Oracle 数据库的优化

CPU

Full MT-hot kernel, scales to 100s of cores and 10,000s of HW threads Support for Critical Threads features in T4 chip 5x performance improvement of high-resolution timer Multi-processing and multi-threading support for next gen database

Memory

Large Page support Optimized Shared Memory (OSM) NUMA I/O Framework Fast DB Restart Latency-aware kernel memory allocator (x86, SPARC) Re-architecture of Virtual Memory sub-system Userland Fast-Memory Registration and Shared Protection Domain

File System Userland file system for DB, Oracle File Server support

I/O

uDAPL, RDSv1, RDSv3, SDP: Support for low-latency Infiniband protocols Direct I/O with concurrent writes Exclusive-IP zone support for RDSv3 to support DBaaS Dynamic reconfiguration for IB HCAs

Key: In Solaris 11 New in Solaris 11.1


冰山一角

更多的 Solaris 针对 Oracle 数据库的优化


Observability Enhanced observability for segmentation faults Read-out of libdtrace by next gen database

Reliability and Availability

Dynamic reconfiguration notifications for DB for resources rebalancing FMA callback for bad hardware

Performance

Improved PGA performance Kernel lock acceleration for Oracle RAC Message Passing Co-processor Remote Memory Access (RMA)

Multi-tenancy Zones: Secure isolation, lowest latency virtualization

Security Transparent crypto off-load


冰山一角

Solaris 针对 Java 的优化


CPU

User-level high resolution timer support WLS scalability, Single-thread mode smt pause() to optimize busy waits in the JVM Java-optimized Solaris scheduling class Fused compare-and-branch with no delay slot New block initializing store (BIS instruction)

Memory Large Page support by JVM T4 2GB pages for Java performance

I/O SDP: Support for low-latency Infiniband protocol HA for SDP

Security Integration with Solaris crypto offload engines (Java 7u4) Zones support for SDP Zones: Secure isolation, lowest latency virtualization

Observability DTrace plugin in Java Mission Control


全堆栈动态故障侦测与恢复

Solaris 针对高可靠性的优化

50% LESS DOWNTIME

Avoid unplanned system downtime with automatic fault management Hardware Fault Recovery

Single Node Service Restart

Heartbeat Traffic Protection

Multi-Node High Availability

Highly Available Applications

World’s Fastest DB Restart

Multi-Site Disaster Recovery


Solaris 针对云计算的优化

Horizon Cloud Management

Zones & Kernel Zones

Nova Compute Virtualization

Elastic Virtual Switch

Neutron Cloud Networking

ZFS File System

Cinder/Swift Cloud Storage

Unified Archives

Glance Image Deployment


却不知国外大型用户正从 Linux 转向 Solaris

在你以为人们正从 UNIX 转向 Linux 的时候

替换掉了:

HP x86

Red Hat Linux

VMware

每天高10倍的客户交易笔数

同样的Licnese数量换来2倍的性能

“We moved to SPARC systems for 3 main reasons: Improved performance, reliability and scalability, and great integration features of Oracle Solaris 11 and Oracle Database.”

T4-4

Large European Telco Company


你可能只看见某些公司曾经从 Solaris 转向 Linux

• Mxxxxx Sxxxxxx 曾是最早从 Solaris 转向 Linux 的大用户之一

• 但是在大约 3 年前，他们的一个管理员说服了高层，把他们的一个原本运行在Linux/X86 上的云服务迁移到了 Solaris/X86

• 结果是：

– 1/6 的硬件就撑起了之前的业务量，或者说，同样的硬件处理能力提升了 6 倍

– 数据通过 ZFS 压缩至了原本的 1/3.5，或者说，同样的存储可以装载 3.5 倍的数据量

– 应用性能提高了15至60倍

– 系统问题降低了40%

– 问题解决时间提速了32%

– 而这位管理员现在已经是公司的 VP 了

28

但你却不知道他们中的一些又从 Linux 转回了 Solaris

http://milek.blogspot.com


Engineered for Reliability

• Most fault tolerant chassis and thermal designs of any x86 server

• Storage and I/O firmware hardened from extreme usage in engineered systems and ZFS Storage Appliance

Extreme Testing/Beyond Certification

• 60,700 test hours of Solaris on Oracle x86 each bi-weekly build

• Oracle x86 used by the Solaris development and QA teams as the x86 development platform

29

Co-Engineered

• More advanced fault diagnosis by Solaris running on Oracle x86 resulting in self-healing and greater uptime

• Oracle Solaris designed to work with Oracle ILOM

Solaris on Oracle x86 - Extreme Reliability


COMPLETE.

SDN Virtualization OpenStack OS


SPARC 芯片

31


5 Processors in 4 Years

SPARC @

SPARC T3 16 S2 cores, 128

Threads 40 nm technology 1.65 GHz

SPARC T4 8 S3 Cores, 64

Threads 40nm Technology 3.0 GHz

SPARC T5 16 S3 Cores , 128

Threads 28nm Technology 3.6 GHz

SPARC M5 6 S3 Cores, 48

Threads, 48MB $ 28nm Technology 3.6 GHz

SPARC M6 12 S3 Cores, 96

Threads, 48MB $ 28nm Technology 3.6 GHz

2010 2011 2012 2012 2013


SPARC @

SPARC Enterprise Supercomputer

“K Computer” PRIMEHPC FX10 Fujitsu M10 server

Die

Processor SPARC64 VII+ SPARC64 VIIIfx SPARC64 IXfx SPARC64 X

Frequency 3.0GHz 2.0GHz 1.85GHz 3.7GHz

Number of

cores 4 cores/CPU 8 cores/CPU 16 cores/CPU 16 cores/CPU

Process 65nm 45nm 40nm 28nm

2010 2011 2012 2013


128 Strands per Socket

SPARC T5 Processor

Features

– 16 S3 cores, 16-128 Strands @

3.6Ghz

– Single or multi-threaded operation per

core

– System scalability to 8 sockets

SPARC Core S3

– 1-8 Strand Dynamically Threaded

Pipeline

– ISA-based Crypto-acceleration

8MB Shared L3$

Integrated I/O

– Double I/O bandwidth over T4

– 2 x8 Lane PCIe 3.0 @ 8GT/s

System Scalability

– 7 Coherence Ports for scalability to 8

Sockets

Power Management

– Dynamic Voltage Frequency Scaling


SPARC T5 CPU Block Diagram

8 threads per Core

IO

Subsystem

BoB BoB BoB BoB BoB BoB BoB

DDR3 – 1066 MHz

Memory

Control

Co

her

en

cy 4

x4 S

wit

ch

Memory

Control

Memory

Control

Memory

Control

8 x 9 Crossbar (~1TBps bandwidth)

C0 C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C15

L3$ B0 L3$ B0

1MB,16-way

L3$ B0 1MB,16-

way

L3$ B2 1MB,16-

way

L3$ B1 1MB,16-

way

L3$ B3 1MB,16-

way L3$ B0 L3$ B0

1MB,16-way

L3$ B4 1MB,16-

way

L3$ B6 1MB,16-

way

L3$ B5 1MB,16-

way

L3$ B7 1MB,16-

way

128 KB L2$

16 KB L1I$

16 KB L1D$

FGU

Crypto

SPARC S3

Core

BoB

DDR3 – 1066 MHz DDR3 – 1066 MHz DDR3 – 1066 MHz

Coherence Unit Coherence Unit Coherence Unit Coherence Unit Link 0

Link 1

Link 2

Link 3

Link 4

Link 5

Link 6

Coherency Links

12.8 Gbps per lane -

12 lanes per link

2 x

8 P

CIe

3.0

@ 8

GB

ps

16

GB

ps

each

dir

ecti

on


Inter-CPU Coherency Overview

• Glueless 1-hop scaling to eight sockets – Glueless means no hub required to connect all 8 CPUs

• A precise directory tracks all L3s in the system – striped across all processors – stored in on-chip SRAMs – flexible for different socket counts

• Higher BW efficiency than snoop-based protocols enables better scaling – 50% more effective bandwidth than comparable

snoopy implementation

36

The First True 8-Way Glueless Architecture Processor

T -

T5 T5

T5 T5

T5 T5

T5 T5

128 Cores System


Maximum 128 Cores, Just Same as T5-8

Inter-CPU Connection of P780

一跳通讯仅在同一个BB模块单元内发生

会出现大量的、不同BB模块之间的2跳3跳通讯

对真实应用会带来非常明显的性能影响

所以，P780满配性能不能够以低配性能指标简单做乘法来得出

P770, P795也同样如此


竞争对手不愿意测试其满配置服务器的关键所在

Oracle独有的 8 路直通互联

以T5服务器满配性能向下线性推导低

配性能，所得值略低于实际性能。

以P7/P7+服务器低配性能向上线性推

导高配性能，所得值远高于实际性

能。

这就是为什么有的厂商一直热衷于

测试低配，然后线性推导高配性能

这就是为什么需要提醒用户谨慎考察

厂商给出的“每核性能”

性能

CPU核数

理想的线性

Oracle T5 Servers

18M P Servers


SPARC 服务器

39


Product Line Overview

SPARC T5 Servers

SPARC T5-1B SPARC T5-2 SPARC T5-4 SPARC T5-8

Processor SPARC T5 3.6GHz SPARC T5

3.6GHz

SPARC T5

3.6GHz

SPARC T5

3.6GHz

Max Processor Chips 1 1, 2 2, 4 4, 8

Max Cores/Threads 16, 128 32, 256 64, 512 128, 1024

DIMM Slots 16 32 64 128

Max Memory 256GB 1TB 2TB 4TB

Drive Bays 2 6 8 8

I/O Slots 2 x PCIe 2.0 EM,

2 NEM,1 FEM slots

8 LP x8 PCIe 3.0,

4 x 10GbE ports

16 LP x8 PCIe 3.0,

4 x 10GbE ports

16 LP x8 PCIe 3.0,

4 x 10GbE ports

Form Factor/RU Blade Rack 3RU Rack 5RU Rack 8RU


TPC-C公测指标对比

SPARC T5 ——最快的 OLTP 数据库服务器

SPARC T5-8 IBM Power 780 IBM Power 780

3-node cluster IBM Power 595 IBM x3850 X5

Processor Model (CPUs/Cores/Threads)

3.6 GHz

SPARC T5

(8/128/1024)

4.14 GHz

Power 7

(2/8/32)

3.86 GHz

Power 7

(24/192/768)

5.0 GHz

Power 6

(32/64/128)

2.40 GHz Intel

Xeon E7-8870

(4/40/80)

tpmC 8,552,523 1,200,011 10,366,254 6,085,166 3,014,684

Response Time (s) (New Order / 90th%)

0.41 0.694 2.1 1.69 0.5

Price / tpmC $0.55 USD $1.38 USD $1.38 USD $2.81 USD $0.59 USD

tpmC / CPU 1,069,065.4 600,005.5 431,927.3 190,161.4 753,671

• P7+ 机型始终拒不参测，是否因为性能提升羞于见人？ • 参测P780最高可配置16个CPU，但实际却只用2个CPU测单机性能，且通过TurboCore技术再关闭16核中的

8核以提高主频，并试图诱骗用户基于此来线性推算满配性能。 • 参测P780最高可配置128核，仅用2台就可以达到256核，但实际却用3台半配来作集群，是否意味着做集群的效果还要好过把P780配置得更高？

• IBM测试所得的响应时间比Oracle T5-8差很远，分别差出69%与4.1倍。

http://www.tpc.org/tpcc/results/tpcc_results.asp?orderby=hardware


TPC-H公测指标对比

SPARC T5 ——最快的 OLAP 数据库服务器

IBM最害怕测TPC-H

IBM测TPC-H用的数据库既不是Oracle，也不是DB2，居然是……Sybase

TPC-H@3TB

System Performance

(QphH) Database

Operating System

Type of CPU Total of cores

SPARC T5-4 Server 409,721 Oracle Database 11g

R2 Enterprise Edition

w/Partitioning

Oracle Solaris

11.1 Oracle SPARC

T5- 3.6 GHz 64

SPARC T4-4 Server 205,792 Oracle Database 11g

R2 Enterprise Edition

w/Partitioning

Oracle Solaris

11 11/11 Oracle SPARC

T4 - 3.0 GHz 32

IBM Power 780 Model

9179-MHB 192,001

Sybase IQ Single

Application Server

Edition v.15.4 AIX Version 7.1

IBM POWER7

- 4.10 GHz

TurboCore 32 (8 8coreCPU)

http://www.tpc.org/tpch/results/tpch_result_detail.asp?id=113060701&layout=











SPECjEnterprise2010公测指标对比

SPARC T5 ——最快的中间件服务器

Oracle Oracle Cisco IBM

Java EE Server 1 x SPARC T5-8 5/8 SPARC T5-8 2 x Cisco UCS B440 M2

1 x IBM Power 780

CPU Chip / Core / GHz (Total)

8 chips, 128 cores, 3.6 GHz SPARC T5

5 chips, 80 cores, 3.6 GHz SPARC T5

8 chips, 80 cores, 2.4 GHz Xeon E7-4870

8 chips, 64 cores, 3.86 GHz POWER7

Ejobs 57,442.17 36,571.36 26,118.67 16,646.34

Ejobs / CPU 7180.27 7314.27 3264.83 2080.79

• SPARC T5-8 服务器比同为 8 CPU的IBM Power 780要快3.5 倍。 • 即使采用IBM热衷使用的极不合理的线性推导，P780满配置性能也仅为T5-8的58%不到。 • 结论：128核的Oracle T5-8远远强于128核的IBM P780

http://www.spec.org/jEnterprise2010/results/jEnterprise2010.html


M6 Big Memory Machine: Terabyte Scale Computing

Unprecedented Performance and Scalability

Best platform for in-memory computing 32 TERABYTE SYSTEM MEMORY

3 TERABYTES PER SECOND

SYSTEM BANDWIDTH

1.4 TERABYTES PER SECOND

MEMORY BANDWIDTH

1 TERABYTE

PER SECOND I/O

BANDWIDTH

384 cores, 3,072 threads, virtualization built in: deploy any size workload


Fujitsu M10 Server Lineup

Fujitsu M10-4

- Max 4 sockets - Max 64 cores - 3.4GHz and 2.8GHz - Rack mount (4U)

Fujitsu M10-4S

- Max 64 sockets - Max 1024 cores - Max 32 TB memory

- 3.7GHz and 3.0GHz

- Rack mount (4U per unit)

- Building Block Architecture

Fujitsu M10-1

- 1 socket - Max 16 cores - 3.2GHz and 2.8GHz - Rack mount (1U)

• Profit from extreme cost savings: • Opex & Capex

• Boost business performance

• Do more with less


SPARC 路线图

46


5 Year Trajectory

Cores 4x Threads 32x Memory Capacity 16x Database TPM 40x Java Ops Per Second 10x

T-Series + 2x Throughput

M-Series + 20%

M-Series +6x Throughput

+1.5x Single Strand

T-Series +1x Throughput

+3x Single Strand

M-Series +2x Throughput

>1x Single Strand

T-Series +2.5x Throughput

>1x Single Strand

SPARC +2x Throughput

+1.5x Single Strand

Solaris 11 Update High-Availability

Memory Scalability

Virtualization

Solaris 11 Software Lifecycle

Scalability, Networking

Security

Solaris 11 Update System Management

IO Scalability

Solaris 11 Update Core Scalability

Solaris 11 Express Software Lifecycle

Scalability

Networking

2010 2011 2012 2013 2014 2015

2010 SPARC Server Roadmap


Solaris 11.x Solaris 11.1 Solaris 11.x Solaris 11.x Solaris 12

Solaris 11

T4 +1x Throughput

+5x Thread Strength

M5 +6x Throughput

+1.5x Thread Strength

M6 +2x Throughput

>1x Thread Strength

In Test

M & T Series +2x Throughput


T5 +2.5x Throughput


2011 2012 2013 2014 2015 2016

Delivered

Software in Silicon • Database Query • Decompression • Encryption • Application Data

Integrity • Java Optimization • Low Latency

Clustering

2018

Oracle Numbers Optimization

M & T Series +1.3x Throughput


Solaris 12.x

2017 2019

Software in Silicon Enhancements

• Database Query+ • Java Optimization+ • Encryption+ • Low Latency

Clustering+

Core Enhancements

Increased Cache

Increased Bandwidth

Solaris 12.x

Next Gen Core

Planned 2014 Oracle SPARC Processor Roadmap


总结

52


“不管黄猫黑猫，只要捉住老鼠就是好猫。”

– 1962年，一些农村地区为了应对饥荒和自然灾害，自发产生了包产到户、责任田等各种各样的生产形式。这些变化，在党内引起较大争论。7月2日，中共中央书记处开会讨论“包产到户”问题。邓小平认为，哪种生产形式能够比较容易、比较快地恢复和发展农业生产，就采取哪种形式。他引用刘伯承经常说起的四川谚语：“不管黄猫黑猫，只要捉住老鼠就是好猫。”

53


Oracle X86 产品线

54

Oracle SPARC 产品线

Oracle 从不排斥 X86，但也一直坚定发展 SPARC


• 重视用户需求，强调用户体验。

• 理性看待互联网行业应用特点与企业应用特点的类似与区别，切勿盲目跟风。

• 切勿本末倒置，不要为了去小机而去小机，也不要为了云计算而云计算。

• 从上层建筑出发，从应用/数据库入手考察需求与利弊。

• 围绕应用/数据库运行性能、可靠性以及使用、管理特性等探讨选型。

• 重视总拥有成本。

• 确保开放性，保证用户随时在X86与SPARC间互移植的可能性与便利性。

• Solaris在SPARC/X86之间保持原代码兼容性

55

Oracle 对选型问题的解答是：

芯片与x86芯片 - oracle.com · latency-aware kernel memory allocator (x86, sparc)...

Documents