Experiences with the use of Open Source Software in
an Operational Environment
Riga 07.04.2005 – Baltic IT&T Forum 2005
Roar SkålinDirector of information technology,
met.no
Norwegian Meteorological Institute met.no
Norwegian Meteorological Institute – met.no
• Responsible for meteorological observations and weather forecasts for Norway, the adjacent sea areas and Svalbard
• HQ in Oslo, regional offices in Tromsø and Bergen
• Three stations in the Arctic: Jan Mayen, Bear Island and Hopen
• Five airport offices• Staff: 465• Operational production 24 hours a day, 365 days a year
The meteorological station at Bear Island
Norwegian Meteorological Institute met.no
The Meteorological Production Chain
• Measurements of the state of the atmosphere (observations)
• Global exchange of observations
• Computer simulations of the future state of the atmosphere (forecast models
• Interpretation by forecasters
• Presentation and distribution
Norwegian Meteorological Institute met.no
met.no Production Chain 1998
Message SwitchTandemIn-house SW
HPC SystemCray UnicosConsortia SW
Control SystemSgi IrixConsortia and In-house SW
MeteorologicalWorkstationIrix/WindowsIn-house and3rd party SW
DistributionIBM Aix/SGI IrixOSS and In-house SW
Data archiveSgi Irix3rd party SW
Data collection
Aix/Windows3rd party andin-house SW
Norwegian Meteorological Institute met.no
Pros and Cons of Open Source Software
+ Shorter ”mean-time-to-repair”+ Reduced investment and operating costs+ Attractive employer for IT-personnel+ In-house competence and eager employees
– Migration cost and time– High degree of freedom– High dependence of in-house competence
0 Quality of software0 Support of software
Norwegian Meteorological Institute met.no
Our Strategy• Standardisation of operating systems
– Linux for meteorological production, visualisation and distribution, for research and for office support (when feasible)
– MS Windows for administrative services, some observation systems and office support (when necessary)
• Standard applications and development framework– Evaluate OSS and use OSS in combination with in-house,
consortia and 3rd party software when appropriate
• Meteorological models– Consortia based software– Encourage OSS and insist on open standards
• Administrative services and observation systems– 3rd party software– Encourage OSS and insist on open standards
Norwegian Meteorological Institute met.no
Implementation and Migration
• Architecture
• Duplication of all critical servers• Incremental process
– Year 2000 issues– High cost systems– Other systems on demand or when upgrade is required
Servers
Production, Application, Databases Terminal, File
Desktops
Thick Thin Thick
Norwegian Meteorological Institute met.no
met.no Production Chain 2005
Message SwitchLinuxOSS (met.no)
HPC SystemLinuxConsortia SW
Control SystemLinuxConsortia, OSS and in-house SW
MeteorologicalWorkstation
LinuxIn-house and
OSS
DistributionLinuxOSS and In-house SW
Data archiveLinux3rd party SW
Data collection
Linux/Windows
3rd party andin-house SW
Norwegian Meteorological Institute met.no
Examples of Open Source at met.no
• OS: Linux, User Mode Linux (UML)• Desktop: KDE• Languages: gcc, Perl, Python, PHP• Databases: MySQL, PostgreSQL• Web publishing: OpenInteract• Applications: Apache, Tomcat, Bind, Exim,
Autofax, OpenLDAP, CUPS, Nagios• Tools: OpenSSH, OpenSSL, SAMBA, FreeRADIUS• Formats: NetCDF, HDF, XML• Middelware: Corba• Configuration: cfengine, Kickstart
Norwegian Meteorological Institute met.no
Open Source contributions by met.no
• Web-based HelpDesk• Web publishing system: wgen• Meteorological Message Switch: Norcom• Minor contributions to several OSS systems, most
notably the web publishing system OpenInteract
Norwegian Meteorological Institute met.no
Competence
• Use of OSS requires access to relevant competence:– system engineers with a genuine interest in OSS tools
and high ability to continuous learning by doing– system developers with a basic understanding of both
methodology and programming and a genuine interest in OSS
• The management must – be able to trust an open community (rather than a
commercial company) – have a basic understanding of OSS, including licences
Norwegian Meteorological Institute met.no
Economy
• Cost of Migration– Total cost € 750 000 - work carried out by internal personnel – Most of the cost due to Y2K and other necessary upgrades
• Cost of Operations– Annual maintenance cost (HW and OS) reduced by € 250 000– Increased staff by one system engineer - Annual cost € 75 000
• Cost of Hardware– Cost per server down 80 - 95% since 1999– Due to both lower HW cost and standardisation on Linux– # of servers increased by a factor of 10 with reduced budgets
• Cost of Software– Overall budget reduced by some 20%– Still have licences for some of the main cost drivers: administrative
systems, backup/archiving, Oracle, some Windows and Adobe
Norwegian Meteorological Institute met.no
Economy: Two examples
• The message switch– Cost of operations 1998: € 75 000– Cost of migration (developing a new system): € 75 000– Cost of purchasing new HW: € 7 000– Cost of operations 2005: ≈ € 0
• The Oracle based climate database– Cost of operations 1998: € 75 000– Internal investment in SW based on Oracle: 10 man-
years– Conclusion: Keep Oracle, move the system to Linux– Cost of operations 2005: € 50 000
Norwegian Meteorological Institute met.no
Security aspects of OSS
• Availability– Use stable versions and User Mode Linux (UML)– Stability issues resolved by online documentation/newslists,
help from development teams or locally produced patches– Fixes for problems related to incompatible software or
unsupported software (missing drivers!) normally available from the net
• Integrity– Prior to installing DMZs and UML, we had one occurrence of
misuse of our Linux based ftp-server
• Confidentiality– The software is open by definition– Not experienced any problems related to data
Norwegian Meteorological Institute met.no
Interoperability
• Exchange of data within the meteorological community based on open formats such as HDF, NetCDF, XML, GRIB
• Linux network operating system has not caused any problems
• For administrative systems, formats like doc, ppt and xls reduces flexibility– Standardised on one office system internally
(currently Microsoft Office)– Must be able to receive these formats– Policy on not sending these formats unless
requested by the receiver
Norwegian Meteorological Institute met.no
Conclusions
• Use of OSS has resulted in– Reduced operating cost– High availability– Highly motivated personnel
• In house competence has been highly beneficial and in some cases instrumental
• The combination of– Using OSS in a standardisation process
– Implementation on demand
has reduced the cost and man-time of implementing the OSS strategy