nyt standardiseret serversetup i sas®9.4 og …...the new sas environment dato skal stå her 6...
TRANSCRIPT
Nyt standardiseret serversetup i SAS®9.4 og governance-struktur for SAS® Data Integration StudioMikael D. Aagaard og Ali Chohan15-06-2017
SDC og Business Intelligence and Decision Support (a short 20 minutes intro)
Dato skal stå her
2
• SDC is servicing 120+ Nordic banks.• Core systems.
• Support systems.
• Data warehouse.
• Business Intelligence and Decision Support.• Data warehouse.
• Data delivery.
• Regulatory reporting.
• Accounting.
• Analysis.
• Development and operations in Denmark and Poland.
• External data center.
Dato skal stå her
Our challenges
In 9.3 we had.
• A little of everything, but not in a very structured way.
SAS 9.4 solution (must have):
• Separation of duties.
• Robustness (separate environments).
• Change control.
• Release control.
• Uniform configuration.
• Roll back of code.
• Log of changes.
Thoughts on SAS governance
Dato skal stå her
4
• Adhere to SAS and SDC Best Practise.
• Description of development-, test- and operational roles.
• Description of procedures.
• ”Stick to the rules”.
• Change control to be used on all SAS systems in SDC.
• All ETL code (where practical) to be developed and
maintained on the development server.
Development Test
Server name EDW-SAS-T-CX-01 EDW-SAS-T-CX-02
Types of users Developers, Admins Testers, Admins.
Users can do Developers:
Develop ETL.
Create new ETL job.
Create tables (layout must be agreed
with DBA etc.)
Create folders.
Deploy jobs.
Admins (and educated dev):
Create servers.
Create libnames
Add/modify users.
Create external connections.
Create SPK packages.
Admins :
Import SPK packages.
Create servers.
Create libnames
Add/modify users.
Create external
connections.
Create folders.
Deploy jobs.
Create SPK packages.
Setup flows.
Testers:
Deploy jobs.
Run flows
User Project
repositories
All code must be developed in user’s project repository.
Check out/Check in
D:\SAS\Config\Lev1\SASMeta\Metadat
aServer\MetadataRepositories\Projects.
Not Applicable
SAS 9.3 setup
Dato skal stå her
5
• SAS 9.3.
• One physical machine for Operation, Development and Test.
• Data storage on Teradata
• ”Environments” separated via LEV in SAS. LEV1 = Production, LEV2 = Development etc.
• ”Environments” can have/have mutually large impact on each other.
• Security is a challenge (both in OS and SAS).
• Promotion or easy solutions?
• Alternative change control!
• Setup is not uniform across ”environments”.
The new SAS environment
Dato skal stå her
6
• Aligned with good SAS and SDC practise.
• 4 Separate environments : Development, Test, Pre-Production and Production.
• Versioning of code in SAS Data Integration Studio (DI) and in code outside DI.• SVN and Tortoise as these integrate with SAS.
• Simpler setup of autoexec and config by using parameter tables that can be used by all environments.
• Better release and version control by semi- automatic moving of code between environments.
• In line with good practice for separation of environments and separation of duties.
Simpler autoexec and config setup
• Parameter table with one entry for each server controls setup of parameters.
• Autoexec files, config files and setup tables are, almost, the same across environments.
• Macro used for ease of overview in autoexec.• Special demands in Development, i.e. Programs run from DI, need of user specific autoexec.
Dato skal stå her
7
Change control SAS 9.3
Dato skal stå her
8
DI Studio• Placed in different subfolders.• Typically adding a date to distinguish versions.• Almost impossible to compare versions.
Open code (OS)• Multitude of ”versioning methods”.• Placed along active code.• Not easily compared.
No Who, When, Where and Why!
Change control in SAS 9.4- Data Integration StudioDato skal stå her
9
• SVN server running on development server.
• SVN is integrated into DI.
• Compare current code with earlier versions.
• Re-install earlier code if needed.
• Code can be checked out and locked by developer.
• Somewhat primitive
Change control outside DI
Dato skal stå her
10
• Tortoise SVN integrates with the OS as as SVN does in DI.
Better release and change control(Promotion)
Create folder structure in Dev environment.
•Release folder created by automated process.
•Initiated in a web browser.
•Structure 100% aligned with original structure.
Dato skal stå her
11
Better release and change control
Place code ready for Test env.
•All new or modified objects (Code, tables, libnames etc.) to go in Test or Production is copied/moved (using SPK) to release folder.
•Done by developer (or other responsible person) when object is ready for testing.
• The move from Development can only be performed by selected personnel.
Dato skal stå her
12
Export package
Import package
Better release and change control
Export code for Test or Prod environments.
• Export release by selecting the desired release.
• Add comment if necessary.
• A description of the contents is saved with the package.
• Log shows result of the export.
• SASPlatformObjectFramework\9.4\
• VA used for analysis.
Dato skal stå her
13
Better release and change control
Select release/package to import
•Select the release to import.
•Select environment to import to.
•Release is imported to the correct location in the metadata folder tree.
Dato skal stå her
14
Summary
To Do• Automatic backup before eksport/import.
• SAS Relationship Reporter to check if all objects are present in packages.
Open issues• How do we handle emergency corrections and fixes in a controlled manner?
• Some objects, i.e. tables, add (1) to the name when moved within the repository, how do we avoid this?
Key points• SAS and SDC practises are guidelines.
• A simpler and safer setup.
• Same autoexec and config on all environments controlled by parameters.
• No specific environment settings.
• All code under version control.
• Version control integrated into SAS DI.
• Same version control used on macros and open code.
• A simpler way of identifying code to be moved. I.e. the developer move the code to the ”same” folder in the release folder.
• A standard way of moving releases that ensures that the code is always placed in the correct location.
• All dialogue via simple web based windows.
• Full documentation of export/import.
• Detailed log for export/import.
Dato skal stå her
15
