copyright © 2002, sas institute inc. all rights reserved. what’s new in sas 9 rick langston sas...

Copyright © 2002, SAS Institute Inc. All rights reserved.

What’s New in SAS 9

Rick LangstonSAS Institute Inc.

Platform R&D Division

[email protected]


Outline The Billy Ts

So many things, so little time

ANYDATE

MP/Connect

DATA step

Threads and PROC SUMMARY


Outline continued ODS

Release Schedule

Summary (not PROC SUMMARY)


Billy T. ?


William T.?


Four Pillars

Usability

Scalability

Interoperability

Manageability


The Intelligence Value Chain


SAS ® 6.09 Usability Experience…circa 1990


SAS ® Usability Experiences Today


Usability – SAS ® 9SAS® Web Report Studio


Usability – SAS ® 9SAS® Enterprise Guide 2.0

Compatible withSAS V8 & SAS 9

New analytic tasks

More controls New Graph styles New extensibility

features


Import XML documents

to SAS ® data sets Export SAS ® data sets

to XML documents XML Map extensions

add flexibility & power to schema definitions

Java ® GUI maps XML to SAS ® data & vice-versa Java-based SAS ® XML Mapper client

Interoperability in SAS® 9SAS ® XML Mapper


So many things... Not enough time for everything!

SQL improvements

OLAP

SPDS

SAS Management Console

Metadata Servers


“ANYDATE” informats ANYDTDTE, ANYDTDTM, ANYDTTME

Handles most acceptable input d/t/dt informats

mdy ambiguities handled


1 data _null_;2 input x: anydtdte20.; list; put x=date9.; cards;

x=01JAN20023 01jan2002x=31DEC20024 12/31/2002x=01MAY20025 may2002x=01JAN20026 01jan2002:12:34:56x=01JAN20027 2002-jan-01x=01JAN20028 2002001


11 options datestyle=mdy;12 data _null_;13 input x: anydtdte20.; list; put x=date9.; cards;

x=02JAN200314 01/02/03x=31DEC200215 12/31/2002x=31DEC200216 31/12/2002x=19MAR195517 55/03/19x=19MAR195518 19/03/55 19 run;20


21 data _null_;22 input x: anydtdtm20.; list; put x=datetime.; cards;

x=01JAN02:12:34:5623 01jan2002:12:34:56x=01JAN02:00:00:0024 01jan2002x=01JAN60:12:34:5625 12:34:56


28 data _null_;29 input x: anydttme20.; list; put x=time8.; cards;

x=12:34:5630 01jan2002:12:34:56x=0:00:0031 01jan2002x=12:34:5632 12:34:56


New in Connect MP connect is not new, but we are still “getting

the word out”

PIPES are new….

– hook your job steps together without using work files


signon mp1 sascmd="!sascmd";signon mp2 sascmd="!sascmd";

rsubmit mp1 wait=no;libname temp 'c:\temp';data newbig; set sashelp.class; do index = 1 to 10000; rand = put(100000 * uniform(123), Z6.0); output; end; run;proc sort data=newbig out=temp.bigsort; by rand;run;endrsubmit;


rsubmit mp2 wait=no;libname temp 'c:\temp';data newbig2; set sashelp.class; do index = 1 to 10000; rand = put(100000 * uniform(567), Z6.0); output; end;run;proc sort data=newbig2 out=temp.bigsort2; by rand;run;endrsubmit;

waitfor _all_ mp1 mp2;

signoff mp1;

signoff mp2;


SAS9 Common BI Infrastructure

SAS Servers

Open Metadata (OMS) Workspace Server Stored Process Data Storage

SAS Foundation Services

System Mgmt InformationUser EventConnection

SAS Query and Reporting Services

Business Metadata ReportingQuery

JNI – Interfaces

Windows Based BI Tools

SAS®Enterprise

Guide

SAS®Information

DeliveryPortal

SAS®AppDevStudio

Dev Env.

SAS® WebReport Studio

SAS® Information Map Studio

Java Based BI Tools

SAS® ETL StudioSAS® Management Console

Portal

Re-usable COM ComponentsSAS Java Components

SAS®Add-in forMS Office


SAS 9.1.3 Shipping!


libname temp 'c:\temp';data temp.merged; merge temp.bigsort(in=mp1) temp.bigsort2(in=mp2); by rand; if mp1 and mp2;run;

proc datasets library=temp kill; run; quit;


signon mp1a sascmd="!sascmd";signon mp1b sascmd="!sascmd";

signon mp2a sascmd="!sascmd";signon mp2b sascmd="!sascmd";

rsubmit mp1a wait=no; libname pipe1 sasesock ":pipe1"; data pipe1.newbig; set sashelp.class; length rand $8; do index = 1 to 10000; rand = put(100000 * uniform(123), Z7.0); output; end; run;endrsubmit;


rsubmit mp1b wait=no; libname pipe1 sasesock ":pipe1"; libname pipe2 sasesock ":pipe2"; proc sort data=pipe1.newbig out=pipe2.bigdata1; by rand; run;endrsubmit;

rsubmit mp2a wait=no; libname pipe3 sasesock ":pipe3"; data pipe3.newbig; set sashelp.class; length rand $8; do index = 1 to 10000; rand = put(100000 * uniform(123), Z7.0); output; end; run;endrsubmit;


rsubmit mp2b wait=no; libname pipe3 sasesock ":pipe3"; libname pipe4 sasesock ":pipe4"; proc sort data=pipe3.newbig out=pipe4.bigdata2; by rand; run;endrsubmit;


libname pipe2 sasesock ":pipe2";libname pipe4 sasesock ":pipe4"; data merged; merge pipe2.bigdata1(in=mp1) pipe4.bigdata2(in=mp2); by rand; if mp1 and mp2;run;

signoff mp1a;signoff mp1b;

signoff mp2a;signoff mp2b;


New in the DATA step– New Functions

– Perl Regular Expressions

– DATA step Hash Tables


String Concatenation Functionscall symput('pat' ||

trim(left(put(total,BEST12.))) ||

trim(left(put(bycount,BEST12.))),

'pattern'||

trim(left(put(total,BEST12.))) ||

trim(left(put(bycount,BEST12.))) ||

' c=pink v=solid ;');

call symput(cats('pat', total, bycount),

cats('pattern', total, bycount,

' c=pink v=solid ;'));


String Concatenation Functions

a = cat("x ", 12.3, " y");

b = catt("x ", 12.3, " y");

c = cats("x ", 12.3, " y");

d = catx("/","x ",12.3," y");

/* "x 12.3 y" */

/* "x12.3 y" */

/* "x12.3y" */

/* "x/12.3/y" */


Sorting Valuesdata s;

array v[50];

set a;

call sortq(of v1-v50);

data s;

array v[50];

set a;

call sortq(of v50-v1);


Median and Percentile y = median(5,8,0,3,4);

/* y is 4 */

lower_quartile = pctl(25,2,4,1,3);

/* lower_quartile is 1.5 */


PUT to LOG == PUTLOGdata _null_;

file names;

input lastName $;

put lastName;

if lastName =: 'J' then

putlog 'NOTE: J last name, ' lastName;

datalines;

Harris

Jones

;


#1 SASWARE Ballot ItemInteger Ranges

data a;

array arrA[10] (1:10);

array arrB[10];

retain arrB (2*1:5);

if x in (1 2 5:10) then

put 'X in range';

if x in arrA then

put 'X in arrA';


Perl Regular Expressions

Search for prior character one or more times.

Search for one or more “A”s.

Search for a digit character.

Search for a word character, alphanumeric plus _.

Search for character in the range between A and Z.

+

A+

\d

\w

[A-Z]

Search for an “A”.

Search for a “B”.

A

B

Typical start and end delimiter./


Perl Regular Expressions

/abc/ xyzabc123 xyzab123c

/abc+\d/ xyzabcccc5123

xyzabcz


Data Validation Validate U.S. phone numbers that look like:

(919) 677-8000

Regular Expression:

/$[2-9]\d\d$ ?[2-9]\d\d-\d\d\d\d/


Data Validation – The Codedata _null_;

if _N_ = 1 then do;

retain re;

re = prxparse("/$[2-9]\d\d$ ?[2-9]\d\d-\d\d\d\d/");

end;

input first last phone & $16.;

if ^prxmatch(re, phone) then

putlog "NOTE: Invalid, " first last phone;

datalines;

Thomas Archer (919)319-1677

Lucy Mallory 800-899-2164

Tom Joad (508) 852-2146

Laurie Jorgensen (252)152-7583

;


Data Extraction Use parentheses to group part of the regular

expression:

/$([2-9]\d\d)$ ?([2-9]\d\d-\d\d\d\d)/

Area CodeSubmatch #1

Local Phone NumberSubmatch #2


Data Extraction – The Code

re = prxparse("/$([2-9]\d\d)$ ?([2-9]\d\d-\d\d\d\d)/");

...

if prxmatch(re, phone) then do;

call prxposn(re, 1, pos, len);

areacode = substr(phone, pos, len);

call prxposn(re, 2, pos, len);

localphone = substr(phone, pos, len);

end;


Search and Replace s/(\w+) \1 /\1 /


Search and Replace – The Code

data _null_;

x = 'it it happened at at SUGI';

re = prxparse('s/(\w+) \1 /\1 /');

call prxchange(re, -1, x);

put x=;

run;

/* x=it happened at SUGI */


Hash Table – Creating and Adding Data

declare Hash ht();

ht.defineKey("ISBN");

ht.defineData("author", "title");

ht.defineDone();

ISBN="0679722769";

author="Joyce"; title="Ulysses";

rc=ht.add();


Hash Table – Finding Data

ISBN="0679722769";

rc = ht.find();

if rc = 0 then

putlog "Found " author title;


Hash Table – Loading from a Datasetdata purchased_books;

if _N_ = 1 then do;

declare Hash ht(dataset: "books");

ht.defineKey("ISBN");

ht.defineData("author", "title", "price");

ht.defineDone();

end;

length author title $ 64 price 8;

set purchased_ISBNs;

if ht.find() = 0;

run;


This looks a lot like… SET with KEY=

MERGE with BY

Custom Format with PUT

data m;

set ids;

set data key=id;

data m;

merge ids data;

by key;

data m;

set ids;

data = put(ids, $LOOKUP.);


Hash Table – PerformanceReal Time to Perform Lookup

Hash Table 3.43s

Load and Search 3.43s

MERGE with BY 9.18s

Sort Data Sets 7.58s

Search 1.60s

SET with KEY= 47.85s

Build Index 3.06s

Search 44.79s

Format and PUT() 22.84s

Build Format 16.03s

Search 6.81s


Hash Tables – V9 Experimental So send in your

comments!

[email protected]

Embryonic…

Java Object Class


PROC SUMMARY and threads What are threads?

Dividing the work

PROC SUMMARY groups by classes

Each thread handles classes

Internally a completely new implementation

Example


PROC SUMMARY and threads

[procdata temp; input group value; cards; 2 61 1 1 3 2 4 1 2 2 5 ; proc summary; class group; var value;


Threads Group 1 gets a thread and a buffer of obs

Group 2 gets the same

Each thread computes the statistics

Brought back together for the output


Threads and I/O Will CPU outrun the I/O processing?

No - the other way around

Big Unix machines maximize their I/O buffering

We need threads to keep up with the I/O


More to come for threads SUMMARY, SORT are the first Base apps

REG, GLM in SAS/STAT

More procs will come

Also used in client/server applications


ODS Updates MARKUP/Tagsets

HTML4

Style Templates/DOCUMENT

Page X of Y

Text insertion

PDF enhancements


ODS MARKUP TAGSET templates

YOU have the power

Available in 8.2 experimentally

"ODS MARKUP: The Power of Choice & Change"

by Eric Gebhart

http://www2.sas.com/proceedings/sugi27/p003-27.pdf

also

http://www.sas.com/rnd/base/index-v9-resources.html


Tagsets

Markup definition file– Defines how the markup should be generated

– Defines the tags and how they should be used


SAS-supplied Tagsets

A sampling of some of the available supplied tagsets:

– CSV

– Default (default XML)

– Docbook

– Event_map

– HTML4

– HTMLCSS (Cascading Style Sheets)

– LaTeX

– WML

– and many more!!!


Make your own destination

Don't like the supplied tagsets?

Need a destination that SAS does not supply?– Create your own

– SYLK tagset created by Jack Hamilton

– DDE to Excel by Frank Poppe

See http://www.sas.com/rnd/base/index-early-faq.html


Tagsets – SCSV ExampleProc Template;define tagset tagsets.SCSV;parent = tagsets.CSV;define event header; start: put ";" / if !cmp( COLSTART, "1"); put """"; put VALUE; finish: put """";end;define event data; start: put ";" / if !cmp( COLSTART, "1"); put """"; put VALUE; finish: put """";end;

define event colspanfill; put ";";end;

define event rowspanfill; put ";";end;

end; /* define tagset */

run;

Semicolons instead of commas


Tagsets – SCSV Example

" 1";"Alfred";"M";"14";"69.0";"112.5"" 2";"Alice";"F";"13";"56.5";" 84.0"" 3";"Barbara";"F";"13";"65.3";" 98.0"

Semicolons instead of commas

ODS Tagsets.SCSV file = 'Ex.csv';proc print data=sashelp.class; run;ODS Tagsets.SCSV close;


The Transition is Coming

ODS HTML file = 'Version9.html';

will invoke MARKUP destination with HTML4 tagset in Version 9.1 by default.

– Current 3.2 spec HTML will be accessible by setting an option in the registry.


MARKUP & TAGSETS

Provides ability to have Markup output YOUR way

Will be the default for HTML in Version 9.1

Uses new TAGSET templates– otherwise referred to as Markup definitions

Create your own destinations


ODS DOCUMENT

New ODS destination -- persists output objects in raw format

Reorder

Rearrange

Replay

without rerunning your procedures


ODS DOCUMENTods document name=sales(write);

Title ‘Quarterly Sales By Product’;

proc tabulate data=sashelp.prdsal2

format=dollar12.2;

class product quarter;

var actual;

table product, quarter*actual*sum=‘’;

run;

ods document close;


ODS DOCUMENT tools

Two new tools to manage ODS Documents

Proc Document

Graphical User Interface – ODS Document Window– odsdocument


RTF: Page X of Y

No more having to insert raw RTF code

NOW:

ods escapechar '\'; title 'Hello ' j=r 'Page \{pageof}';

BEFORE:

title 'Hello ' j=r 'Page {\field{\*\fldinst {\b\i PAGE }}}{\b\i of }{\field{\*\fldinst {\b\i NUMPAGES }}}';


TEXT option

Some text BEFORE Proc Report

Name Sex Age Height Weight

Alfred M 14 69 112.5

Alice F 13 56.5 84

Barbara F 13 65.3 98

Carol F 14 62.8 102.5

Henry M 14 63.5 102.5 Some text AFTER Proc Report


TEXT option

ODS RTF TEXT = 'Some text before PROC';

ODS PDF TEXT = 'Some text before PROC';

Actually available in V8.2

Improved to allow style option for formatting


ODS PDF Enhancements

Compressed

MUCH faster

Embedded fonts – definitely 9.1


Other SAS ® 9 FeaturesSoftware Installation

Customized software installations

Formal distribution of setinits via email

Setinit transparently applied by the install program

Improved “silent” install for Windows installations

New InstallCenter on www.sas.com


Other SAS ® 9 FeaturesShipping & Packaging

New installation Kit

CD packaging is 3-tiered

All products shipped,making add-ons easy


Other SAS ® 9 Features

Section 508 CompliantSAS Windowing Environment

Output Delivery System

National Language SupportJapanese, Chinese, KoreanFrench, German, Italian, Polish, Hungarian

SAS Display Manager in Kanji


9.0

9.1

9.1.2

9.1.3

9.2

2004

9.0Q4

2002 2003

9.1Q4

9.1.23/31

9.1.36/30


Not carried forward from V8 Windows 95/98

CMS

VAX/VMS

32-bit HP, Solaris, AIX

Mac

Intel ABI

MIPS ABI


Summary Four Pillars (The Billy Ts)

ANYDATE

MP/Connect

DATA step

Threads and PROC SUMMARY

ODS

Release Schedule

LOTS OF GOOD STUFF!!!