fully automated updating of arbitrarily complex excel
TRANSCRIPT
![Page 1: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/1.jpg)
Fully Automated Updating of Arbitrarily Complex Excel WorkbooksDavid OesperLands’ End
WIILSUJune 28, 2017
![Page 2: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/2.jpg)
The Challenge: Automation
• A coworker left the company and I inherited a set of his programs that he ran manually at the end of every fiscal month.
• He generated a series of data sheets which he then manually cut and pasted into existing Excel workbooks.
• After all that, he manually emailed the revised Excel workbook to several recipients.
![Page 3: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/3.jpg)
The Challenge: Excel
• The Excel workbooks my coworker was updating were not simple grids of rows and columns.
• Our clients who received these Excel workbooks designed them and preferred not to make any changes to the design.
• It was evident to me that SAS alone would not be able to reproduce these workbooks exactly (at least not without a lot of work!).
![Page 4: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/4.jpg)
The Challenge: A SAS Solution?
• I knew how to add hidden data sheets and add references to them on the main worksheet.
• But could SAS, running on Linux, make a “strategic strike” on an existing Excel workbook, and update or replace existing data sheets?
• If I could update the workbook, then emailing the revised workbook and/or uploading it to a Windows directory would be easy.
![Page 5: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/5.jpg)
Five Step Approach
• Create a static version of your Excel workbook to use as a “model” for SAS, and
• Add one or more hidden data sheets
• Clone a copy of your Excel workbook for editing
• Replace your data sheets “on the fly”
• Save or email to the destination(s) of your choice
![Page 6: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/6.jpg)
What You’ll Need
• SAS 9.4 TS1M2 or later
• SAS/ACCESS to PC Files
• Microsoft Excel (to prepare your model Excel workbook with data sheets)
![Page 7: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/7.jpg)
Step #1
Create a static version of your Excel workbook to use as a “model” for SAS
![Page 8: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/8.jpg)
Excel Workbook to UpdateU.S. Census Divisions – Population by State
![Page 9: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/9.jpg)
Add one or more hidden data sheets
Step #2
![Page 10: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/10.jpg)
Excel Workbook to UpdateU.S. Census Divisions – Population by State
![Page 11: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/11.jpg)
Excel Workbook to UpdateOur Data Sheet
proc print data=sashelp.us_data;title 'sashelp.us_data';var statename population_2010
division;run;
![Page 12: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/12.jpg)
Excel Workbook to UpdateU.S. Census Divisions – Population by State
Data!B4
![Page 13: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/13.jpg)
Excel Workbook to UpdateOur Data Sheet
proc print data=sashelp.us_data;title 'sashelp.us_data';var statename population_2010
division;run;
Data!B4
![Page 14: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/14.jpg)
Excel Workbook to UpdateU.S. Census Divisions – Population by State
![Page 15: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/15.jpg)
Clone a copy of your Excel workbook for editing
Step #3
![Page 16: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/16.jpg)
Copy Excel Workbook for EditingPC SAS
/* Today's date in yyyymmdd format: example 20170503 */%let tdate = %sysfunc(today(),yymmddn8.);
/* Path and File Name of model Excel workbook */%let xlinpath = C:\Users\David\Documents\SASIN;%let xlinfile = U.S. Census Divisions.xlsx;
/* Path and File Name of output Excel workbook */%let xloutpath = C:\Users\David\Documents\SASOUT;%let xloutfile = U.S. Census Divisions &tdate..xlsx;
![Page 17: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/17.jpg)
Copy Excel Workbook for EditingPC SAS
/* Copy the Excel Workbook */options noxwait noxsync;data _null_;
call system(%unquote(%bquote(')copy "&xlinpath\&xlinfile"
"&xloutpath\&xloutfile"%bquote(')));
run;options xwait xsync;
![Page 18: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/18.jpg)
Copy Excel Workbook for EditingLinux SAS
%let userid = daoespe;
/* Today's date in yyyymmdd format: example 20170503 */%let tdate = %sysfunc(today(),yymmddn8.);
/* Path and File Name of model Excel workbook */%let xlinpath = /sasdevr/devr180/&userid;%let xlinfile = U.S. Census Divisions.xlsx;
/* Path and File Name of output Excel workbook */%let xloutpath = /sasdevr/devr7/&userid;%let xloutfile = U.S. Census Divisions &tdate..xlsx;
![Page 19: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/19.jpg)
Copy Excel Workbook for EditingLinux SAS
/* Copy the Excel Workbook */data _null_;
/* change to input directory */call system("cd &xlinpath");/* copy model Excel workbook to output directory */call system("cp –f %unquote(%bquote(')
&xlinfile%bquote(')) &xloutpath");/* change to output directory */call system("cd &xloutpath");/* rename excel file on output directory */call system("mv %unquote(%bquote(')
&xlinfile%bquote('))%unquote(%bquote(')
&xloutfile%bquote('))");run;
![Page 20: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/20.jpg)
Replace your data sheets “on the fly”
Step #4
![Page 21: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/21.jpg)
Getting Our Updated Census DataScoop .csv file off a web site: filename url
2016 U.S. population estimate
![Page 22: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/22.jpg)
Getting Our Updated Census DataScoop .csv file off a web site: filename url
filename newpop url"https://www2.census.gov/programs-surveys/popest/datasets/2010-2016/state/asrh/scprc-est2016-18+pop-res.csv";
2016 U.S. population estimate
You could also use PROC HTTP to get the data.
![Page 23: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/23.jpg)
Getting Our Updated Census Data
Existingdata sheet
SAS data set for new data sheet
Excel row #s off by 1 -Column headers take a row
![Page 24: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/24.jpg)
Create an Updated Excel WorkbookUpdate Working Copy of Excel File!
/* use LIBNAME XLSX access method */libname xlout xlsx "&xloutpath/&xloutfile";
/* update data sheet with a merge */data xlout.'Data'n;
merge xlout.'Data'n sasuser.newpop;by statename;
run; This is the Key!!LIBNAME XLSX
+Simple DATA Step MERGE!
![Page 25: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/25.jpg)
SAS Analytics Pro – PC SASDoes Not Include SAS/ACCESS to PC Files
/* update data sheet with a merge */data xlout.'Data'n;
merge xlout.'Data'n sasuser.newpop;by statename;
run;
SAS LOGlibname xlout xlsx "&xloutpath/&xloutfile";
ERROR: The product with which the engine XLSX is associated is either not licensed for your system or the product license has expired. Please contact your SAS installation representative.ERROR: Error in the LIBNAME statement.
![Page 26: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/26.jpg)
Updated Excel Workbook2016 Population Estimates
Data sheet was hidden in the original workbook – remains hidden
![Page 27: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/27.jpg)
Save or email to the destination(s) of your choice
Step #5
![Page 28: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/28.jpg)
Copy Updated Excel WorkbookFrom Linux to Windows
/* filename same as output libname */filename xlout "&xloutpath/&xloutfile";
/* filename for FTP to Windows drive */filename ftpout ftp
"DIRMARKET124/BA/Staff/Dave/SASOUT/&xloutfile"recfm=s host='********' user='********‘pass='********';
![Page 29: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/29.jpg)
Copy Updated Excel WorkbookFrom Linux to Windows
/* copy the file byte-for-byte */data _null_;
length filein 8 fileid 8;filein = fopen('xlout','I',1,'B');fileid = fopen('ftpout','O',1,'B');rec = '20'x;do while(fread(filein)=0);
rc = fget(filein,rec,1);rc = fput(fileid,rec);rc = fwrite(fileid);
end;rc = fclose(filein);rc = fclose(fileid);
run;
![Page 30: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/30.jpg)
Email Updated Excel WorkbookFrom Linux
/* output filename using the email accessmethod - notice we are emailing the Excelworkbook we created on Linux */
filename eml emailto = ('[email protected]')cc = ('[email protected]')type = 'text/plain‘emailid = '[email protected]'attach = ("&xloutpath/&xloutfile"ct="application/octet-stream" ext="xlsx")subject = 'Updated U.S. Census Data';
![Page 31: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/31.jpg)
Email Updated Excel WorkbookFrom Linux
/* Send email, but only if no errors */%macro sendemail;%if %length(&syserrortext) ne 0 %then %return;data _null_;
file eml;put "Here are the updated population estimates
for each state from the U.S. CensusBureau." /;
put 'Best Regards,' /;put 'Dave';
run;%mend sendemail;%sendemail;
![Page 32: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/32.jpg)
Email Updated Excel WorkbookFrom Linux
![Page 33: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/33.jpg)
Email Updated Excel WorkbookFrom Linux
![Page 34: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/34.jpg)
Email Updated Excel WorkbookFrom Linux
![Page 35: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/35.jpg)
Summary
• We used the LIBNAME XLSX engine, a part of SAS/ACCESS to PC File Formats, to update data sheets in an Excel workbook using a simple DATA step MERGE.
• We used FILENAME URL to “scoop” a .csv file off of a web site to get our updated data.
• We used FILENAME FTP to copy our updated Excel workbook from a Linux server to a Windows directory.
• We used FILENAME EMAIL to email our updated Excel workbook to a recipient list.
![Page 37: Fully Automated Updating of Arbitrarily Complex Excel](https://reader036.vdocuments.site/reader036/viewer/2022081503/62a2aefae80d16245c007225/html5/thumbnails/37.jpg)