scalla developer summary xrootd /cmsd andrew hanushevsky slac national accelerator laboratory cern...

Scalla Developer Scalla Developer SummarySummary

xrootd /cmsd

Andrew HanushevskySLAC National Accelerator Laboratory

CERN Workshop10-November-08

http://xrootd.slac.stanford.edu

10-November-08 2: http://xrootd.slac.stanford.edu

Supported Platforms

FreeBSD New addition (best-effort support only)

Linux i386_linux24, i386_linux26, x86_64_linux_26

MacOS ppc_darwin_70, ppc_darwin_80, x86_darwin_90

Solaris sun4x_58, sun4x_59, sun4x_510, sunx86_510

Windows XP (client only)


Supported Compilers

g++ Up to version 4.3.2 Used for FreeBSD, Linux, and MacOS

icc Used for Itanium architectures

SGI Linux

Sun CC Always used for Solaris compilations


Supported Build Environments

In-House configure.classic Works on all platforms without problems Missing some features

Make install

Autotools make Supported by Derek Feichtinger Has some issues on certain platforms

E.G., Unable to build XrdPosixPreload.so 32/64 bit compilation/linking on 64/32 bit platforms


General Source Layout I

XProtocol/ cmsd and xrootd protocol data structures

Xrd/ Protocol driver for cmsd and xrootd (networking and scheduling)

XrdAcc/ Default access control (i.e., authorization)

XrdBwm/ Bandwidth manager plug-in

XrdCS2/ Castor plug-in (obsolete)

XrdClient/ Client related code (e.g., TXNetFile, xrdcp, etc)

XrdCms/ Cluster Management Services protocol plug-in

XrdCns/ Cluster Name Space daemon and other components


General Source Layout II

XrdCrypto/ Cryptography support classes for security plug-ins

XrdMon/ Monitor data collection agent

XrdNet/ All networking related classes (used by everyone)

XrdOdc/ Open Distributed Cluster support (deprecated)

XrdOfs/ Open File System plug-in

XrdOlb/ Open Load Balancing protocol implementation (deprecated)


General Source Layout III

XrdOss/ Open Storage System plug-in

XrdOuc/ Object Utility Classes (used by everyone)

XrdPosix/ Posix compatibility libraries

Includes preload library

XrdPss/ Proxy plug-in

Used as a storage system plug-in


General Source Layout IV

XrdSec/ Authentication protocol driver plug-in

XrdSecgsi/ Gsi-based authentication plug-ins

XrdSeckrb4/ Kerberos IV authentication plug-in

XrdSeckrb5/ Kerberos V authentication plug-in

XrdSecpwd/ Password authentication plug-in

XrdSecsss/ Simple Shared Secret authentication plug-in

XrdSecunix/ Unix (i.e., NFS-like) authentication plug-in


General Source Layout V

XrdSfs/ File System Interface definition

Includes the default implementation

XrdSut/ Security utility classes

XrdSys/ OS-dependent classes (used by everyone)

XrdTokenAuthzOfs/ ALICE ofs plug-in implementing ALICE security

XrdVersion.hh Holder to identify the version everywhere

XrdXr/ Proxy plug-in (obsolete, to be deleted)

XrdXrootd/ Xrootd protocol plug-in


So, What’s A Plug-In?

Run-Time Loadable Code A single class based on an abstract interface

Plus an extern ‘C’ object instantiator

Always packaged as a shared library Configuration file specifies location of library

Automatically loaded Objects created as needed

Extends xrootd or cmsd functionality Avoids massive rebuilds and code branching

The base system uses pre-defined statically linked plug-ins


lfn2pfnprefix encoding

Storage System(oss, drm/srm, etc)

authentication(gsi, krb5, etc)

Clustering(cmsd)

authorization(name based)

Storage System(Flash Based System)

File System(ofs, sfs, alice, etc)

Protocol (1 of n)(xrootd, cmsd)

Scalla Plugin Architecture

Protocol Driver(Xrd)


The Plug-Ins

Dynamic plug-in always packaged as a shared library Authentication plug-in

Always as libXrdSec.so Others as libXrdSecxxxxxxxx.so

Authorization plug-in Currently, no plug-ins exist (default statically linked-in)

File System plug-in Available: libXrdOfs.so

Storage System plug-in Available: libXrdProxy.so (provides proxy access)

Protocol plug-in PROOF team uses a plug-in to implement PROOF protocol

External Management Interface plug-in (XMI) Used by Castor to interface cmsd to Castor name space


Closer Look At Security Plug-Ins

xrootd has no security FALSE!FALSE!

By default, security is not enabled This simplifies setup for most sites

You must configure security to get it Not difficult but yet another thoughtful step

Once enabled, xrootd is as secure as you want So, the myth is now busted!So, the myth is now busted!


What Is Security?

Two phases to security Authentication

Done in the xrootd/cmsd protocol layer Multiple simultaneous authentication modes supported

Authorization Done in the ofs filesystem layer Basic user/group/netgroup capability list mode available But, authorization is a plug-in

Can substitute any other mode you’d like


xrootd Authentication

Multi-protocol design Server provides available protocols Client chooses one of the possibilities Each protocol name comes with configuration data

Allows client to self-configure for the protocol

Each protocol implemented as a plug-in Easy to add new protocols


Currently Available Protocols

gsi Grid certificate based authentication

Extension gsiGMAPLDAP maps DN to username via LDAP

krb4 Standard Kerberos IV

krb5 Standard Kerberos V

pwd Hidden password authentication

sss Simple shared secret

unix Basic Unix NFS-like [non-]security


Authentication Protocol Plug-Ins

Each protocol is a shared library plug-in libXrdSecxxxxxxxx.so

The xxxx xxxx is the protocol name (e.g., libXrdSeckrb4krb4.so)

The plug-ins are managed by libXrdSec.so Authentication protocol client/server driver

Finds and loads appropriate shared libraries Uses abstract security interface for all interactions

Credential generation & authentication


Abstract Class XrdSec

class XrdSecProtocol {public:

virtual int Authenticate( XrdSecCredentials *cred, // In XrdSecParameters **parms, // Out XrdSecClientName &client, // Out XrdOucErrInfo *einfo=0 ) // Out

virtual XrdSecCredentials *getCredentials( XrdSecParameters *parms=0, // In XrdOucErrInfo *einfo=0 ) // Out

virtual const char *getParms( int &psize, // Out const char *host ) // In};

• C++ framework to manage protocols as plug-ins • Generic protocol (XrdSecProtocol)

• Protocol implementations inherit from XrdSecProtocol

server

client

Courtesy ofCourtesy ofGerri GanisGerri Ganis


libXrdSec.so

• libXrdSec.so provides the Protocol Manager - Server: instantiated at start-up from configuration file:

- load protocol plug-ins that server can / wants to run- binds (subsets of) the list to hosts or host patterns

- controls authentication mode by host - Client: build-up list loading protocols the first time needed

- library loaded only if authentication is required• Plug-in implementations provide a public instantiator to create an instance of the protocol• Simple negotiation: list of allowed protocols sent to the client, who chooses the one to try first

Courtesy ofCourtesy ofGerri GanisGerri Ganis


XrdSec Implementation Is Generic

• Depends only on network and utility modules • XrdNet, XrdOuc, and XrdSys

• Can be easily used in a non-XROOTD context

• Working example of standalone client and server programs using XrdSec available at

http://ganis.home.cern.ch/ganis/ROOT/SECURITY/testXrd.tgzCourtesy ofCourtesy ofGerri GanisGerri Ganis


Authentication Architecture

login

Send credentials

Send client-specific security configuration

libXrdSec.solibXrdSec.so

Get Get CredentialsCredentials

(Select Protocol)(Select Protocol)

Get SecurityGet SecurityConfigurationConfiguration

Multiple exchanges allowedMultiple exchanges allowed

libXrdSecgsi.so

libXrdSeckrb4.so

libXrdSeckrb5.so

libXrdSecpwd.so

Dynamically selected by clientServer specifies availability

Libraries managed by libXrdSec.soLibraries managed by libXrdSec.so

Config File

1

2

3 n

authenticate

Ask for more

get credentials 0


Authenticated Identity

Passed to file system layer to be used for authorizationPassed to file system layer to be used for authorization

char prot[8]; // Protocol usedchar *name; // Entity's namechar *host; // Entity's host namechar *vorg; // Entity's virtual organizationchar *role; // Entity's rolechar *grps; // Entity’s groupschar *endorsements; // Protocol specific endorsementschar *tident; // Trace identifier (do not modify)void *cert // Pointer to certificate void *cert // Pointer to certificate (future)(future)int clen; // Length of certificate int clen; // Length of certificate (future)(future)

XrdSecEntity.hhXrdSecEntity.hh


Why Do It This Way?

Can implement almost any model needed Without changing any server/client code at all

Simplifies security audit procedures

Can quickly evolve as requirements change And support different modes for different moods

SLAC Atlas uses sss authentication Fermi (a.k.a. GLAST) uses unix authentication But generally many experiments don’t want anything

Only for reading but not if they are writing to xrootd


But Wait!

Where is SSL? SSL applies to the transport layer

The security framework applies to the protocol layer Recall, any protocol can optionally use this framework

A transport protocol does not easily give you any options

And SSH? You really mean using ssh keys, don’t you?

No one wrote a plug-in for that yet


Let’s Recap

xrootd/cmsd are all about plug-ins Expands applicability within the design focus Allows you to piggy-back new functionality

For instance, PROOF

Are all points pluggable? Just the obvious ones

There is always room for improvement here E.G., MSS plug-in now part of the oss plug-in

So, let’s see talk about writing plug-ins


Writing Plug-Ins I

Read the documentation in the plug-in “hh” XrdAccAuthorize.hh

Authorization abstract interface XrdCmsXmi.hh

XMI abstract interface XrdOss.hh

Storage System Abstract Interface XrdProtocol.hh

Protocol abstract interface XrdSecInterface.hh

Authentication abstract interface XrdSfsInterface.hh

File system abstract interface

Look at an existing plug-in of the same type


Writing Plug-Ins II

General things All plug-ins must be thread-safe Avoid high-latency actions

This generally causes pile-up/melt-down Usually because of thread starvation and timeouts

Some interfaces allow background processing High latency is not an issue then

Use existing classes in the repository libCrypto.a, libCryptoLite.a, libXrdNet.a, libXrdOuc.a, and libXrdSys.a


Writing Plug-Ins III

For a plug-in to be included in the CVS repository Must follow naming conventions Adhere to the abstract interface

Interface changes are rare and always backward compatible Be stand-alone

Cannot rely on frameworks and external add-ons This includes STL

• A sore point but it’s saved us countless hours of debugging Have a good Makefile and Makefile.am Compile and run on all supported platforms Usually have detailed documentation

See existing references (e.g., Security) http://xrootd.slac.stanford.edu/doc/sec_config/sec_config.htm

Otherwise, we can simply reference your plug-in web page


Packaging

Official releases are available via web site http://xrootd.slac.stanford.edu/

We are working on getting a better more generic URL Currently, Wilko Kroeger cuts official releases

This does not preclude special integrated releases ALICE Castor PROOF Root

These usually suffer a little drift from the official release

We are working on making the CVS available r/o The fastest way is an AFS accessible directory

/afs/slac.stanford.edu/public/software/scalla CVS head checked out for viewing and a gtar file of the same

Planning for a web interface to the repository


Licensing & Contributions

Currently, Scalla is under a BSD License http://www.opensource.org/licenses/bsd-license.php

May change to Apache 2.0 license http://www.opensource.org/licenses/apache2.0.php

We may need to restrict how names are used cmsd, Scalla, and xrootd

This is to prevent confusion relating to derivative works

Contributions happily accepted Must conform to licensing requirements Handled in the Linux tradition


SLAC Support

Memo of understanding…The SLAC science program is heavily dependent on xrootd. I can thereforeThe SLAC science program is heavily dependent on xrootd. I can thereforeassure you that xrootd will be maintained by SLAC for at least five years.assure you that xrootd will be maintained by SLAC for at least five years.As you are aware, xrootd is an open source product and will remain freelyAs you are aware, xrootd is an open source product and will remain freelyavailable.available.

I believe that xrootd is brings valuable and currently unique capabilitiesI believe that xrootd is brings valuable and currently unique capabilitiesin scalable high-volume data analysis. It is part of SLAC's mission toin scalable high-volume data analysis. It is part of SLAC's mission toencourage wide use of developments like xrootd where they can benefitencourage wide use of developments like xrootd where they can benefitnational and international science programs.national and international science programs.

SLAC staff supporting xrootd will be encouraged to examine, as time permits,SLAC staff supporting xrootd will be encouraged to examine, as time permits,problems and suggestions submitted by users who are not connected with theproblems and suggestions submitted by users who are not connected with theSLAC program. Non-trivial work in response to such submissions wouldSLAC program. Non-trivial work in response to such submissions wouldrequire that work be also beneficial to SLAC's use of xrootd.require that work be also beneficial to SLAC's use of xrootd.

I particularly encourage a collaborative approach to maintaining andI particularly encourage a collaborative approach to maintaining anddeveloping products like xrootd. This approach promotes wide use, anddeveloping products like xrootd. This approach promotes wide use, andcreates a situation where effort spent on issues raised by a collaboratorcreates a situation where effort spent on issues raised by a collaboratorwould be considered valuable to SLAC by default.would be considered valuable to SLAC by default.

Richard MountRichard Mount


Active Developers

Current Active Software Developers Andreas Peters (Andreas.Joachim.Peters cern.ch)

Castor/xrootd Andrew Hanushevsky (abh stanford.edu or Andrew.Bohdan.Hanushevsky cern.ch)

Server Bertrand Bellenot (Bertrand.Bellenot cern.ch)

FreeBSD issues, Windows Derek Feichtinger (Derek.Feichtinger cern.ch)

Autotools Fabrizio Furano (Fabrizio.Furano cern.ch)

Client Gerardo Ganis (Gerardo.Ganis cern.ch)

Classic make, Cross-Platform issues, Security and PROOF Tofigh Azemoon (azemoon slac.stanford.edu)

Monitoring Wilko Kroeger (wilko slac.stanford.edu)

MPS scripts, packaging, and release issues


Getting Support

Currently available venues Official web site (always check there first)

http://xrootd.slac.stanford.edu/ General problem mailing list ([email protected])

Must be subscribed (see http://xrootd.slac.stanford.edu/xrootdlist.html)

CERN Web Site http://project-arda-dev.web.cern.ch/project-arda-dev/xrootd/site/index.html

Request Tracker (RT) problem system at SLAC This is still experimental

E-Mail the right developer Actual bugs, contributions, and enhancement requests

User support An evolving issue for experiments and groups

E.G., OSG provides 1st level support for VDT Developers generally cannot provide direct user support


Acknowledgements

Software Contributors CERN: Derek Feichtinger, Fabrizio Furano, Andreas Peters Fermi: Tony Johnson (Java)

Root: Gerri Ganis, Bertrand Bellenot SLAC: Jacek Becla, Tofigh Azemoon, Wilko Kroeger

Operational Collaborators BNL, INFN, IN2P3

Partial Funding US Department of Energy

Contract DE-AC02-76SF00515 with Stanford University

scalla developer summary xrootd /cmsd andrew hanushevsky slac national accelerator laboratory cern...

Documents

infile system pluginavailable

xrootd networking

neededextends xrootd

code branchingthe base

space daemon

derek feichtingerhas

abstract interfaceplus

alice securityxrdversion