1

Learnings from My Migration to EG 7.1 Kent Lewandowski, Kaiser Permanente, Oakland, CA

ABSTRACT SAS ® Enterprise Guide (EG) introduced a whole new way to manage and execute SAS projects. It has many unique and useful features: an interactive log, “process flow” interface, branch run control, and starting with version 7.1, smart highlighting and the ability to track source code changes. EG version 7.1 was released at the end of 2014, yet in my enterprise, it is only now being deployed. I’ve had the chance to review both versions 6.1 and 7.1. and compare them side by side. Drawing on my experience, as well as past presentations by other experts, I will highlight some of the useful changes in EG Version 7.1, as well as some of what I perceive to be additional opportunities for improvement.

The presentation is intended for users with beginner to intermediate SAS skills who are either somewhat familiar with SAS Enterprise Guide, or else would like to learn more about it.

INTRODUCTION SAS EG has many features to support large and complex projects, for instance, Process Flow, Project Log Summary, Program History Tracking, Program Scheduling, Support for Shared / Grid Computing, and Program Editor tools. For many SAS users whose projects are ad-hoc and require relatively low amount of resources, the incentive to abandon BASE SAS in favor of SAS EG is low because they don’t feel there is a need to leverage these features. However, recent changes in enterprise computing may cause you to reconsider working solely in BASE SAS:

• IT departments need to centralize computing resources and streamline product licensing and upgrades, resulting in drive to use SAS in a shared environment instead of on individual PCs

• Running out of space on your PC to store SAS data sets, coupled with availability of shared computing resources / Grid computing that support Enterprise Guide but not Base SAS

• Need / desire to leverage new tools to manage increasingly complex analytical assignments

• Need / desire to leverage new features and tools available in SAS EG 7.1 to become more a productive programmer

This paper seeks to highlight some of the important features of Enterprise Guide, as well as to provide you some useful tips and learnings from my experience moving from EG 6.1 to 7.1. At the end, I provide a few suggestions to SAS for improving Enterprise Guide in future versions.

SHORT HISTORY OF ENTERPRISE GUIDE

SAS Enterprise Guide first became available with version 1.0 in late 1999. Since then it has undergone many changes. EG 6.1 was released in July 2013, along with Base SAS version 9.4. Enterprise Guide 7.1 was released in October 2014. While EG 6.1 includes many enhancements over prior versions, EG 7.1 includes some notable improvements and new features not previously available. I have attempted to document these below. The first section focuses on improvements in EG 7.1, while the following section highlights new features.

IMPROVEMENTS IN VERSION 7.1

User Interface Improvements

1. Display of Datasets improved. You can now double click in the libraries list view (lower left) and contents of the dataset are displayed on-screen. While this doesn’t sound like much, it is a very useful improvement. In EG 6.1, the dataset is added to Process Flow when you click it, but contents are not shown on-screen. Then you must hunt around in your Process Flow to find the dataset.

Learnings from My Migration to EG 7.1, continued

2

Display 1. In EG 6.1, when you click “AUDITTABLE” data set in “Server List” on the left-hand side, the dataset is added to Process Flow, but contents are not shown on-screen. You must locate it in the process flow (right hand side) and double click there. If your Process Flow is large, this is time-consuming.

Display 2. In EG 7.1, when you click the same data set in Server List, the dataset contents are immediately shown on-screen.

2. Copy and Paste between Projects is improved. Copy and paste between projects in SAS EG 6.1 only works for individual data sets and associated program elements. It does not work for “branches” (steps in your Process Flow that must run in sequential order.)

Learnings from My Migration to EG 7.1, continued

3

Display 3. In EG 6.1, you cannot copy a branch from an existing Project into a new one. (“Copy” is greyed out.)

In SAS EG 7.1, this functionality is improved. You can now copy a group of objects as a branch from an existing project into a new one using Copy and Paste.

Display 4. In EG 7.1, after you copy a branch, and open a new project, Paste is allowed.

Productivity Tools

3. Data Step Debugger is improved. This useful feature allows users to view variables in the Program Data Vector as the DATA step executes. EG 7.1 has an interactive debugger environment (click the “Green Bug”.) Unfortunately, our installation of EG 7.1 has not allowed me to test this feature yet. I would encourage readers to

Learnings from My Migration to EG 7.1, continued

4

view the excellent video presentation on the SAS Dummy blog at http://blogs.sas.com/content/sasdummy/2017/08/18/debug-program-history/

4. Code Analyzer enhanced with Expand Macros. The Program Analyzer implemented in EG 6.1 is a handy feature that lets generate a Process Flow from any block of SAS code. In SAS EG 7.1, this feature has been enhanced with “expand macros” to allow you to generate code from your stored processes / macros. Please see “Tips and Tricks” section below for more details on using the Code Analyzer in EG.

NEW FEATURES ADDED IN EG 7.1 1. Project (“EGP”) Search Tool. This is a very useful and new feature developed by Chris Hemedinger from the

SAS Institute. It is available in the upper right corner of your Process Flow window. You can type text from any DATA step or project element, and it will appear in your search results. The search results are also context-sensitive and will open the associated object when you click them.

Display 5. Using the Project Search Tool in EG 7.1.

2. Tracking Program History. EG 7.1 provides new tools to keep track of program history and changes you make to your project. These can be accessed through the Program Menu.

Display 6. You can access changes made to your Project via Program > View All Program History.

Learnings from My Migration to EG 7.1, continued

5

Clicking Program History opens the “Commit History” window showing changes “committed” to your project. If you click a change record in Commit History, the associated Process Flow element and code is shown in the window below.

Display 7. Viewing Commit History for an EG 7.1 Project.

When using the COMMIT tool the first time (which saves all your changes to the Program History tracker), all the previously “uncommitted” elements will be shown, which is a little confusing.

Display 8. The first time you click “COMMIT” for an EG 7.1 Project, all elements are listed.

The next time you COMMIT, however, only the elements you changed since you made the last COMMIT will be shown. Red-green colored highlights exactly what was changed.

Learnings from My Migration to EG 7.1, continued

6

Display 9. Clicking Program > Commit brings up Commit Changes tab. New changes are highlighted in green.

3. New SAS Macro Variable Viewer and SAS System Options Viewer Tools. These tools grew out of a custom task that Chris Hemedinger developed prior to the release of SAS EG 7. They are now both available in the Tools menu. Neither of these tools are available in EG 6.

To view SAS Macro Variables used in your project, click “SAS Macro Variable Viewer.”

Learnings from My Migration to EG 7.1, continued

7

Display 10. Viewing global macro variables via the Macro Variable Viewer.

To view SAS Systems Options in your project, click “SAS System Options Viewer.”

Learnings from My Migration to EG 7.1, continued

8

Display 11. Viewing system options via the System Options Viewer.

TIPS AND TRICKS FOR EFFECTIVE USE OF SAS EG Below are some Tips and Tricks I found to be useful in interacting with Enterprise Guide. Except for the Tip about Committing your changes automatically, all of these tips are able to be implemented by EG 6.1 users. 1. Use “X” to exit the current object instead of switching windows. While this might seem obvious to some, it

took me a long time to learn. Working in the Process Flow can be confusing as it grows (see tips below to keep your Process Flow from getting too large.) If you don’t exit the code object (PROC or DATA step) you are currently looking at, the default view will be the top of your Process Flow. If your code object is buried deeper in the Process Flow, you will have to scroll around to find it.

2. Use the context sensitive help. It’s surprising how much there is available just by hovering over key words.

3. Use “setup” code at the beginning of your projects to assign libraries and global variables. You may

wish to combine this with an “Audit Table” type data set that lets you track previous project executions.

Learnings from My Migration to EG 7.1, continued

9

Display 12. An example of a “setup” program that establishes library references and records activity in the AUDITTABLE data set.

4. Clean your PROJECT LOG every once in before it gets too large and unwieldy. If you are running your project for the first time in a while, clear the project log via Project Log > Clear Log.

Display 13. Project Log in EG 6.1. If you recently ran the same steps, it might be a good idea to clear the log.

5. Break up your Project into Multiple Process Flows. This is a good tip from Chris Hemedinger, author of The SAS Dummy blog. At some point the Process Flow becomes too large and unwieldy as the project expands. Breaking up the project into multiple Process Flows allows better oversight and management, since you don’t have to scroll around so much to find objects. See the next tip about using the Code Analyzer to produce new Process Flows automatically.

Learnings from My Migration to EG 7.1, continued

10

Display 14. Creating a new Process Flow within your Project and renaming it via Properties.

Once you have separated your Process Flows into more manageable chunks, arrange them sequentially in the order you expect to run them:

Display 15. Rearranging Process Flows in order of execution within your Project. In this example, “Setup Project” (renamed “Setup Library”) is moved to the top.

6. Use the Code Analyzer to Build Process Flow! If you simply copy a long section of your old SAS program into a program block in EG, then it will produce many elements, and will clutter up your Process Flow. Instead, use the CODE ANALYZER tool to break up your SAS Code into small steps and de-clutter your Process Flow.

Learnings from My Migration to EG 7.1, continued

11

Display 16. Result of too much logic in a single code block. Use the Program Analyzer (Program > Analyze) to separate this into smaller pieces of code so that your process flow is more horizontal.

The Code Analyzer works the same for both EG 6.1 and EG 7.1, except that EG 7.1 offers “Expand Macro” function.

Display 17. Using the Analyze Program Flow tool in EG 6.1. Highlight a program object, then select Program > Analyze > Analyze for Program Flow.

Learnings from My Migration to EG 7.1, continued

12

Display 18. Next step in Analyze Program Flow (version 6.1). Process Flow Objects are generated from the submitted code. Errors are shown at bottom. Note, you cannot click the errors in Analyze Program to find them in code. It may be advisable to create your new process flow first, and then figure out the errors from there.

You can optionally choose “Create Process Flow” which creates a newly named process flow for you. I recommend to assign a new name to the process flow that results from your analyzer and then incorporate that into your project as a separate task (see above tip re: creating multiple Process Flows.) You can also move these objects back to a different Process Flow. This can be difficult, however. See Desired Enhancements section #3 below.

Display 19. Using Program Analyzer in EG 6.1 creates a cleaner, better looking process flow.

Learnings from My Migration to EG 7.1, continued

13

7. When using Code Analyzer, don’t try to process too much code at once. It is more efficient to put only a limited amount of code through the Analyzer at one time. Otherwise the resulting Process Flow becomes too full of objects and difficult to troubleshoot.

8. Use unique names for test data sets. If you reuse data set names like “DUPS” or “TEST,” the Process Flow will get messed up with cross-links. Instead use “DUPS_XXXX” where XXXX refers to the current data set being analyzed.

Display 20. When producing test data, use unique data set names like DUPS_IP4 instead of “DUPS,” so that your Process Flow does not get cluttered up with cross-links.

SAS also made the recommendation to write temporary / intermediate data sets to a temp library which you de-assign at the end of your program.

9. Create new data sets with each step instead of “reusing” them. Even though we used a unique name for our DUPS data in the PROC SORT step above, the Process Flow still looks cluttered:

A better PROC SORT for EG Process Flow would include the OUT= option and create an intermediate Data Set, like this:

The resulting Process Flow is cleaner:

10. Use Conditional Processing. Conditional processing is a useful feature in Enterprise Guide that lets you avoid having to program macros. While the setup is intuitive, it is sometimes unclear what objects to attach the

Learnings from My Migration to EG 7.1, continued

14

conditions to. After some trial and error, I figured to add the condition to the “parent” object (“Setup Libraries” in the example below). EG then attaches this condition to the “child” objects when it executes the branch.

Display 21. First, prompts are created in “Setup Libraries” via Object Properties (Properties > Prompts.)

In the example below, if the user enters the prompt “NC” or “SC” for “Region”, EG will run “Setup Temp Variables CA.” Otherwise, EG will run “Setup Temp Variables ROC.”

Learnings from My Migration to EG 7.1, continued

15

Display 22. Setting up Conditional Processing. The checkmark indicates which “branch” was executed.

11. Set the Option to “Commit” Program History every time you save your project. This is a tip for EG 7.1 users. See “Tracking Program History“ in the NEW FEATURES section above. By default, you must remember to COMMIT changes to your project. If you forget to do this, you are not able to take advantage of this feature, because Program History is only useful after you created your first “save point.” Therefore, it’s best to simply set Options (Tools > Options) to Save Program History every time you save the project.

Display 23. Setting the Option to “Commit” Program History every time you save your project.

12. When Using Grid Computing, Set the Options at the Beginning of your Project. Grid Computing support is already offered in SAS EG 6. To enable SAS EG to gain better efficiency from Grid Computing, you need to set Project Properties options. Click File > Project Properties > Code Submission

Display 24. Setting Project Properties to use Grid computing and enable parallel execution.

You need to set these properties before you run your project (not mid-way through.) Otherwise EG gets confused and can’t find datasets in WORK library.

There is an Option in EG 6.1 (and 7.1) that allows “Initialize Grid” to occur when you connect to a shared environment. This might be preferable to dealing with Project Properties.

Learnings from My Migration to EG 7.1, continued

16

Display 25. Setting “Initialize grid” option (not checked by default) in Tools > Options.

DESIRED ENHANCEMENTS IN FUTURE EG VERSIONS. Working on this paper has given me ideas for future enhancements in Enterprise Guide. It is possible some of these are already being worked on by SAS.

1. Display the associated LIBNAMES with datasets in the Process Flow. Especially for data sets not found in WORK library, it would be useful to see the corresponding library listed in the dataset name. It would be great if the library associated with the dataset could be placed in the top left corner, e.g. WORK.BUCKETS2S in the example below:

2. Would be Nice if Analyze Program Tool had context-sensitive error display. SAS EG has spoiled us with a rich set of features in the log summary. We can click any Error in the Log and the referenced line of code is brought on-screen. It would be great if this capability were also included in the Program Analyzer tool.

Learnings from My Migration to EG 7.1, continued

17

Display 26. Errors resulting from SAS Code Analyzer are not context-sensitive.

3. Improve Copy and Paste between Process Flows. While copy and paste between EG projects is improved, the same cannot be said for copy and paste of elements between Process Flows. While using EG 7.1, I encountered numerous situations where I could not copy a series of objects in one Process Flow and paste these into another. The “Move” tool, while useful, often resulted in lost objects / elements. Overall the experience felt very clunky. It seems like this is something SAS could improve upon.

CONCLUSION I hope you have learned some things from this paper and my experience using SAS EG. SAS EG 7.1 represents an improvement over EG 6.1 with some User Interface and Productivity improvements like the Data Step Debugger. More importantly, EG 7.1. offers new tools like Tracking Program History, Macro Variable Viewer, and the Project Search Tool. Enterprise Guide is a feature-rich tool that can be configured many ways. Some tips are listed in this paper to help you get the most out of Enterprise Guide and become a more productive programmer. Lastly, some ideas are listed for SAS to include in future versions of their product.

REFERENCES • Malczewski, Matt. June 19, 2015. “What’s New In Enterprise Guide 7.1.” Proceedings of the Toronto Area

SAS Society. http://www.torsas.ca/attachments/File/06192015/Malczewski_EG7_1_June2015.pdf • Purushothaman, Ramya "How May I Help?" The SAS® Enterprise Guide® Analyze Program Feature,

Proceedings of the SAS Global Forum 2013. http://support.sas.com/resources/papers/proceedings13/311-2013.pdf

• Hemedinger, Chris, “New SAS programming features in SAS Enterprise Guide 7.1”. http://blogs.sas.com/content/sasdummy/2014/10/12/eg-71-new-programmer-features/

• Hemedinger, Chris, “Through the years: SAS Enterprise Guide versions.” http://blogs.sas.com/content/sasdummy/2013/08/06/eg-versions-through-years/

• Hemedinger, Chris, “10 tips for organizing your SAS Enterprise Guide projects.” http://blogs.sas.com/content/sasdummy/2011/10/10/10-tips-for-organizing-your-sas-enterprise-guide-projects/

Learnings from My Migration to EG 7.1, continued

18

CONTACT INFORMATION Your comments and questions are valued and encouraged. Contact me at:

Kent Lewandowski Kaiser Permanente 300 Lakeside Drive, 13th Floor Oakland, CA 94612 E-mail: kent.lewandowski@kp.org

SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Other brand and product names are trademarks of their respective companies.