· securefiles – next generation unstructured data management oracle open world - november 2007...
TRANSCRIPT
![Page 1: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/1.jpg)
1
![Page 2: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/2.jpg)
<Insert Picture Here>
SecureFiles – Next Generation Unstructured Data ManagementOracle Open World - November 2007Ravi RajamaniSr. Product Manager
Scott LynnPMTS
![Page 3: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/3.jpg)
3
Agenda
• Enterprise Data• Current Approaches to managing data• SecureFiles: Changing the paradigm• High Performance• Advanced Features• Out-of-the-Box benefits with SecureFiles• Demonstration• National Ignition Facility – Real world experience
![Page 4: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/4.jpg)
4
Enterprise Data
Audio, Video
Vector, Spatial
SQL Data
Documents
Image Data
XML Data
Emails, IM
![Page 5: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/5.jpg)
5
Enterprise Data
• Drivers for unstructured data growth• Increased digitization of content
• Healthcare, Finance, Insurance, Banking• Compliance• Web 2.0• Scientific/Research Community
Semi & Unstructured
80%
Structured20%
Yearly Data Growth (estimates)Structured – 15 – 20%
Unstructured – 50- 200 %
![Page 6: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/6.jpg)
6
Managing Information
• Organizations need to efficiently and securely manage
Structured Semi-Structured Unstructured
XML PDF
• Simplicity and performance of file systems makes it attractive to store file data in file systems, while keeping relational data in DB
• Enterprise applications manipulate both files and relational data• Rich user experience, compliance, business integration
![Page 7: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/7.jpg)
7
Files Belong with Relational Data
• Many applications have both files and relational data• e.g. Document Management, Medical, CAD, Imaging
• This split compromises security, robustness, and management• Disjoint security and auditing models• Changes cannot be made atomically• Backup and recovery are fragmented• Search across relational data and files is difficult• Space management is complicated• Separate interfaces and protocols
• Two data managers for one application is one too many
![Page 8: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/8.jpg)
8
Oracle SecureFilesConsolidated Secure Management of Data
• SecureFiles is a new database feature designed to break the performance barrier keeping file data out of databases
• Similar to LOBs but much faster, and with more capabilities• Transparent encryption, compression, deduplication, etc.• Preserves the security, reliability, and scalability of database• Superset of LOB interfaces allows easy migration from LOBs
• Enables consolidation of file data with associated relational data• Single security model• Single view of data• Single management of data
![Page 9: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/9.jpg)
9
Innovations
• SecureFiles is a major rearchitecture of how the database handles unstructured (file) data• Not an incremental improvement to LOBs
• Entirely new:• Disk format• Network protocol• Caching and locking• Redo and undo algorithms• Space and memory management• Cluster consistency algorithms
![Page 10: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/10.jpg)
10
SecureFiles Vs. NFS
• SQL+ File test, single stream, single host• Using Secure Files is faster across the board
• Upto 2x faster for Queries, 6x for Inserts• Tests run using both SecureFiles and NFS/ext3 in metadata
journaling only (default for NFS)• Filesystem-like performance
File Read Performance File Write Performance
0.010.1
110
100
1 10 100 1000 10000
File Size (KB)
MB
/s
0.1
1
10
100
1 10 100 1000 10000
File Size (KB)
MB
/s
SecureFiles NFS
![Page 11: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/11.jpg)
11
SecureFiles Vs. BasicFiles
050
100150200250300
0.01 0.1 1 10 100
MB
/s
Writes: New Space
File Size (MB)
050
100150200250300
0.01 0.1 1 10 100M
B/s
Writes: Reused Space
File Size (MB)
SQL+ File Test: Concurrent Reads/Writes, OCI, 4 streams • Adding Files using New Disk Space: Up to 2x faster• Adding Files Reusing Space: Up to 22x faster• Reads up to 3x faster
050
100150200250300
0.01 0.1 1 10 100
MB
/s
Read Performance
File Size (MB)
SecureFiles LOBs
![Page 12: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/12.jpg)
12
Breaking Performance Records! – Made for Speed
• Our Lab Setup• 3 EMC CX700 connected through 2
switches, 12 LUNs spanning the 3 arrays
• 4 RAC Nodes• Xeon 3.4GHz, 2 CPUs, each
hyperthreaded, 6GB of memory• HBA is 2Gbs - limits I/O bandwidth
of a node to 276 MB/s • OCI, 8 concurrent streams total.• SecureFiles: Filesystem_like_logging,
NoCache• Provides Sustained Throughput of
• 776 MB/s for File Read• 462 MB/s for File Writes
• Satisfies 38 TB/day of data ingestion rate using off-the-shelf hardware & software
0
200
400
600
800
0.01 0.1 1 10 100
MB
/s
File Size (MB)
SecureFile Reads
SecureFile Writes
High Performance Experiment
![Page 13: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/13.jpg)
13
Advanced Features - Compression
• SecureFiles introduces Compression for unstructured data• Huge storage savings
• Industry standard compression algorithms• 2-3x compression for typical files (doc, pdf, xml)
• Automatically detects if SecureFile data is compressible• Skips compression for already compressed data• Auto-turn off compression when space savings are minimal or zero
• Server-side compression• Allows for random reads and writes to SecureFile data
• Two levels of compression provide different compression ratios• Compression Levels: MEDIUM (default), HIGH• Higher the degree of compression, higher the latency and CPU overhead
incurred
• Part of the Advanced Compression Option
![Page 14: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/14.jpg)
14
Advanced Features - Deduplication
• Enables storage of a single physical image for duplicate data• Significantly reduces space consumption• Dramatically improves writes and copy operations
• No adverse impact on read operations• May actually improve read performance for cache data
• Specially useful for content management, email applications and data archival applications
• Part of the Advanced Compression Option
Secure hash
![Page 15: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/15.jpg)
15
Advanced Features - Encryption
• Extends Transparent Data Encryption (TDE) functionality to SecureFile data• Data encrypted on disk
• Support for industry-standard encryption algorithms• 3DES168• AES192 (default)• AES256
• Unified security level for both file and relational data• Part of the Advanced Security Option
![Page 16: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/16.jpg)
16
SecureFile Interfaces
• SecureFiles can be accessed by both database clients and file system clients
• Database clients use standard LOB interfaces• JDBC, ODBC, OCI, OCCI, .NET, PL/SQL
• File system clients use the file system protocols implemented in the XML DB or Content DB repository• FTP access• WebDav Access• Http Access
![Page 17: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/17.jpg)
17
Out of the Box Benefits
• Easy conversion from LOBs to SecureFiles using Online Redefinition• Existing LOB applications benefit with no changes
• New installations• By setting db_securefiles= ALWAYS
• File system based applications can switch to SecureFiles using Oracle’s native implementation of industry standard files protocols• FTP, WebDAV, Http
• SecureFiles is fully integrated with• XML DB, Oracle Multimedia, Oracle Spatial, Content DB
![Page 18: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/18.jpg)
18
Demonstration
![Page 19: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/19.jpg)
19
On-line Redefinition
• Table (Before Redefinition)create table doc_tab (pkey number(10) not null,2 document clob)3 lob(document) store as doc_tab_document_lobseg
4 (nocache logging retention);
• Redefinition Tablecreate table doc_tab (pkey number(10) not null,
2 document clob)3 lob(document) store as SECUREFILE doc_tab_document_lobseg_sf
4 (nocache logging retention auto COMPRESS DEDUPLICATE);
![Page 20: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/20.jpg)
20
Application Transparency
• Create Tablecreate table doc_tab (pkey number(10) not null,
document clob)lob(document) store as SECUREFILE doc_tab_document_lobseg(nocache logging retention autoCOMPRESS DEDUPLICATE ENCRYPT);
• Create text index on compressed ,deduplicated and encrypted SecureFile
beginctx_ddl.create_index_set('document_iset');ctx_ddl.add_index('document_iset','document');end;/CREATE INDEX document_idx ON doc_tab(document) INDEXTYPE IS
CTXSYS.CONTEXT;
![Page 21: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/21.jpg)
21
National Ignition Facility – Real World Experience
Philip A. Adams
![Page 22: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/22.jpg)
22
The Best of Files and Databases
• SecureFiles have all the leading-edge file system capabilities • Deduplication, Encryption, Compression, Logging, Versioning
• SecureFiles have advanced database features not in file systems• Transactions, Read Consistency, Flashback• Readable Standby, Consistent Backup, Point in Time Recovery• Fine Grained Auditing, Label Security• Sliding Inserts• XML indexing, XML Queries, XPath• Real Application Clusters• Automatic Storage Management• Partitioning and ILM• Search across meta-data and file content
• Capabilities go far beyond any other database or file system• Having the best of both worlds removes the need to compromise
![Page 23: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/23.jpg)
23
For More Information• Oracle Technology Network (OTN)
• http://www.oracle.com/technology/products/database/securefiles/index.html
• SecureFiles Tutorial• http://www.oracle.com/technology/obe/11gr1_db/datamgmt/secure
file/securefile.htm
• Advanced Compression White Paper• http://www.oracle.com/technology/products/database/oracle11g/pdf/advan
ced-compression-whitepaper.pdf
Oracle DEMOgrounds (O07)
Moscone West Exhibition Hall
Demonstration
![Page 24: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/24.jpg)
24
Sessions From Oracle Development
• S291688 - Oracle Database 11g: Next-Generation Performance and Scalability, 4:45 -5:45 pm, Moscone South 104
• S291862 - Improve the performance and manageability of your NFS environment with Oracle Database 11g Direct NFS Client, 11:30 am - 12:30 pm, Moscone South 304
• S291861 – Advanced Compression in Oracle Database 11g, 12:15 - 1:15 pm, Moscone South 304
Monday, Nov 12
Tuesday, Nov 13
Thursday, Nov 15
• S291857 – Next Generation Unstructured Data Management using SecureFiles, 3:15 -4:15 pm, Moscone South 304
• S291857 – Total Recall using Flashback Data Archive, 4:45 - 5:45 pm, Moscone South 304
• S292033 - Oracle Database 11g: Next-Generation Data Management Platform, 10:45 -11:45 pm, Moscone South 304
• S291689 – Using Oracle Database in Amazon Web Services Environment, 1:00 - 2:00 pm, Moscone South 304
![Page 25: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/25.jpg)
25
Demos From Oracle Development
SecureFiles
Advanced Compression
Total Recall
Monday, Nov 12 – Thursday, Nov 15Oracle DEMOgrounds, Moscone West
Recovery Manager (RMAN) and Flashback Technologies
![Page 26: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/26.jpg)
26
Q U E S T I O N SA N S W E R S
![Page 27: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/27.jpg)
27
![Page 28: · SecureFiles – Next Generation Unstructured Data Management Oracle Open World - November 2007 Ravi Rajamani Sr. Product](https://reader034.vdocuments.site/reader034/viewer/2022042201/5ea0adaebbaf4041b56a4233/html5/thumbnails/28.jpg)
28