Application servers and ZOPE

An overview by many authors:

T.Tammet, P.Browning, K.Birch. C.Petrelli…

Plain cgi vs server

• Cgi program: started, runs, stops. Ordinary command-line program.

• Server program: runs all the time, day after day.

• Server program is used by client programs which send connection requests to a port. Server listens to requests, typically creates a new thread for each, answers.

Plain CGI vs application server

• Plain CGI architecture:

Browserhttp url

http serverJust file?

Program?

Get file from thefile system

Run programon the server

Application server

• Possible app server architecture:

Browserhttp url

http server

External databases

Files infile system

External programs

App server

Programsin appserver

databasesIn appserver

Middleware

App server 1

• Essentially, application servers offer an integrated Web development platform that allow connection and management of a variety of enterprise resources such as Web servers, databases, and legacy application systems.

App server 2

• Characteristic of an application server is a three-tiered architecture with an application's development and deployment layer between the client and backend of legacy systems.

• The application server provides a developer with a set of common otools and services that most applications need, allowing a new level of integration among components.

App server 3

• Application servers can link multiple systems and applications over diverse hardware and operating systems.

• Some application servers provide an integrated development environment. This can eliminate the need for multiple tools or code conversion, but it may not allow you to use other tools that you've used previously.

• Application servers can take advantage of reusable modular component models such as CORBA, COM/DCOM, IIOP, or EJB (Enterprise JavaBeans). This means you won't have to reinvent the wheel over and over again.

• Application servers provide access to a variety of databases, either through direct support (i.e., SAP, Notes, CICS, etc.), or standardized interfaces (i.e., JDBC, ODBC, etc.).

App server 4

• With features such as load balancing and pooling and caching, application servers can give you more control over how applications run.

• Application servers provide better reliability using using cluster fail-over, auto-detect and auto-restart, and multiple object instances to insure prompt response to client requests.

• Application servers usually provide numerous management tools including remote and local application management, prototype testing (for use prior to deployment), and system monitoring.

• Security features such as authentication, Secure Sockets Layer (SSL), certificates, access rights, and encryption can be used and integrated by application servers with operating systems and directory services.

• Application servers can allow database-intensive business logic to perform better, lowering network traffic and improving access performance.

CMS

• Content management system.

• Goal: publish and edit web pages EASILY by a workgroup (not just ftp ….)

• Example cases: newspaper www systems,news and discussion sites, portals, …

• Use case: a large site which is changed fairly often.

What is a CMS?

(from Ed Ort, see Rezourzes)

“a CMS is a concept and not a product” Lowndes, pers. comm.

N.B. Only a selection of features is shown in the outer shell

(from Browning & Lowndes, 2001 in. prep.)

CMS Feature Onion

Top 6 (out of 41!) features @ IWMW2000

• Self-service authoring/frictionless publishing for non-technical content providers

• Workflow management – submit, review, approve, archive • Integration with existing data/databases AND user

authentication systems • Roles based security

• Flexible output – author once, publish in many formats

• Metadata management

Does the “orthodox” Web deliver these?

Zope - what is it?

• Middleware ++

• Server Unix, NT or Mac (MacOS X)

• An object database that thinks it’s a filesystem

• Can be entirely browser-driven

• Best “done” rather than described

Zope - For

• Transparency of publishing (Web forms, ftp, http put, WebDAV)

• A growing number of drop-in objects (ODBC, Oracle, MySQL, Confera, Squishdot)

• Open source

• Community

What is Zope?

• A Python Web Application Server • A transactional object database that thinks it’s a file system• It has many elements of a CMS ...•… and quite a lot more besides

Web Client

XML-RPC

WebDAV

FTP Client

Web Server

Z Server

Zope Core

ZODB

Z Classes

Products

RDBMSFile

System

Demo

• Management screens• Zope - a Swiss Army Knife for the Web?

examples

• WorldPilot - a third-party Product

What does Zope compete with?

• ACS - ArsDigita Community System (TCL)

• Enhydra (Java)

• Type3 (PHP)

• Mason (Perl)

• Coldfusion

• MS Site Server (IIS, ASP, …)

Among many, many (> 80) others ….

Where does Zope sit in the Application Server/CMS landscape?

(The descent of WebMan?)

Easing the Webmaster bottleneck

Complexity of applications

ZopeEnhydraTeamSiteSpectra

Apache & File System

FilemakerProActive Server PagesColdFusionPerl & CGI

Zope suited to lightweight solutions too

What are the reasons not to use Zope?

• I need to use Apache

• I have to script in DTML

• I have to author in a browser

• I have to put all my content in one file

• The documentation is poor

• There is no commercial support

• You can’t recruit Python programmers

• It is not Java

What are the reasons not to use Zope?

• I need to use Apache You can• I have to script in DTML Use Python or Perl• I have to author in a browser HomeSite,

emacs, BBEdit, Word, etc• I have to put all my content in one file You don’t• The documentation is poor It’s eclectic!• There is no commercial support DC and others• You can’t recruit Python programmers Maybe• It is not Java Correct

Is Zope a CMS? I

Content Integration

and Versioning

Workflow

Application

User Management

User Interface

DataRepository

Deployment

Is Zope a CMS? II

Content Integration

and Versioning

Workflow

CONTENT INTEGRATION AND VERSIONINGCore SQLMethods, Search Interface, Undo, VersionProducts LocalFS, ZODBCDA, ZOracleDA, ZMySQLDA,

ZopeLDAP, XMLDocument Utilities load_site.py

WORKFLOWProducts Portal ToolKit (PTK)

Is Zope a CMS? II

Content Integration

and Versioning

Workflow

DEPLOYMENTCore ZEOProducts SiteAccess, CacheManager, XronUtilities ReportLab HowTos Running Zope off CD ROM, Build a WAP site with Zope

APPLICATIONSCore DTML, Zcatalog, External Methods, MailHostProducts PythonMethods, PerlMethods, SQLSession, FSSession, ZUBB, Squishdot, Site Summary, Zwiki, WorldPilot, zCommerce

USER INTERFACECore TTW Authoring & ManagementUtilities HS Extensions, RadioUserland, ZopeStud

USER MANAGEMENTCore UserFolder, Users who have users who have users, Roles, Fine-grained permissionsProducts LoginManager, Membership, smbUserFolder, ZopeLDAP,

mysqlUserFolder

DATA REPOSITORYCore Data.fs (ZODB - a transactional object database that thinks it’s a file system), ZEOProducts OracleStorage, BerkeleyStorage, CompressedStorage

What is Zope? Promo answer:

• Zope is the leading Open Source application server, specializing in interactive content such as portals and content management systems.

• Mature system, based on many years of work in application servers

What is inside Zope?

• A Web server built in.

• A Web based interface

• An object database

• Relational database integration

• Scripting language support

Advantages: Customer

• Zope doesn’t just deliver pages to the customer, it provides the ability to engage in interactive, customized content.

• URLs that “Mom” can understand

Advantages: Content Managers

• Everything is through-the-web

• Choose your weapon

• Content re-use, order prevails over chaos

• Flexible searching

Advantages: Developers

• Modular architecture

• Customizable searching

• Rich object model

• Integrated object database

• Extensible security

• Integration of external data

Advantages: Business

• Keeping control.– Your website is the heart-beat of your

electronic presence. If your site isn’t in business, you aren’t in business.

– Unfortunately, the most important thing in the world to your technology vendor is their business plan, not yours.

– Source code, right to fix, right to modify.

Advantages: Business

• Price Advantage– Zope’s commercial competitors operate on a simple

principal: account control. Everything is structured to make you pay a lot and pay repeatedly.

– Competitive software can cost up to $1M just for the licenses, often $100K+ just to get started. Moreover, these other alternatives cash in on hidden charges, such as yearly transaction fees.

– Zope is “free of charge.” Money saved on license feeds can be applied to actually getting a solution, rather than a piece of software.

Simple

• Manageable code base– ~88,000 lines of Python– ~45,000 lines of C

• For comparison PHP4 is ~150K lines of C code

• Built on core Python 1.5.2 distribution

• Python has been described best as “executable pseudo-code.”

Separation of Roles

• Robust security expression

• Differentiation between users, roles and permissions

• Users may have roles only in a “local” context. This is granular to the object level.

Delegation of Control

• Content managers control content

• Developers control logic

• Administrators control server

• You can structure control any way you desire

Integrated Environment

• Object management through-the-web

• Simple interface

• Provides a unified OO interface to both Zope data as well as legacy information

True Web Objects

• Conversion of URLs to objects

• Ability to hide the low-level transport (HTTP) from the developer

• Develop reusable objects which present themselves via multiple methods

Passionate Community

• Over 1,500 members of the mailing lists• Lists have reached saturation point in volume• For reference, 2500/month = over 80/day

0

500

1000

1500

2000

2500

Jan Mar May Jul Sep Nov

Zope-Dev

Zope

Scalability

• > 1,000,000 hits/day on commodity hardware

• Compares favorably with other application servers

• Tries to play friendly with caching

• Commercial solutions for high-scale situations (Zope Enterprise Option)

Architecture

Apache, IIS,Netscape

Browser Program Login

Python

PCGI/FastCGI

FTPHTTP/

WebDAVMonitor

XML-RPC/SOAP

ZServer/Z Publisher

Framework

IMAPSMTP

LDAP

ZODBRDBMS

Other?

Filesystem RDBMS

UltraSeek

http/https

Others...

Zope Management Model

• Zope is designed to be managed “through the web”

• A few operations (e.g. adding products) require disk access

• Content is not “strewn” around the file system

Zope Management Interface

• Two-paned UI like Windows Explorer– Navigator (on the left)– Workspace (on the right)

• Views along top

• HTML 3.2 with/some JavaScript• Available at:

http://server.com/. . ./manage

Roles for Managing Zope

• Sneak peek to security:– Using Zope management interfaces requires

that the user authenticate as a user with the Manager role

– Immediately following installation there is one (1) user defined in the access file in the Zope home directory

– By definition this user has all roles

Zope Management Interface

• Logging In– Authentication initially controlled with

username and password stored in access file– access file stored in the root of Zope

installation– formatsuperuser:{alg}password

• Superuser has all roles!

Authenticating to /manage

• Accessing the /manage interfaces will challenge the user or a password using Basic Authentication– Password is in clear text– Can be used in conjunction with Secure Sockets

Layer (SSL) for more security

• Although you don’t type it in each time, login is transmitted on each request

Logging In

• Basic authentication dialog box:

Zope Management InterfaceN

avig

ator

Worksp

ace

Zope UI Structure

• Frames-based interface– Navigator — Structure (i.e., the tree)– Workspace — Contents (i.e, the leaves)

Zope Management Navigator

• Hierarchy of folders

• Root folder name determined at installation– Could be /, /Zope,

/anything_you_want

Zope Management Workspace

• Top of the frame offers one or more views behind a corresponding tab

• Displayed tabs vary depending on what you are browsing (e.g., document, folder, etc.)

Zope Management Workspace

• For folders:– Contents

– Properties

– Import/Export

– Security

– Undo

– Find

– Help

Folder Properties

• Properties are attributes of a folder

• Useful for– Content management

– App development

• Data types– string, boolean, date,

float, int, lines, long, etc.

Folder Import/Export

• Move folders– across Zope instances– backups– product publishing

• Import must be done from the file system ($ZOPE_HOME/import)

Import & Export

Folder Security

• Manage existing role assignments

• Create new role definitions

• New roles are then available for assignment to users below the definition!

Folder Security

• Note that each product introduces its own security behaviors that are unique to its function

• Example– Specific ZSQL Methods security

Folder Security

Folder Undo

• Exposes transactions since last database pack• Stores object, method, user and time• Undo will not violate object integrity

Folder Find

• Where did I use that property?• What have I tinkered with since yesterday?• Expensive

Adding Things

• The Available Objects– Contains available

objects that can be added in the current context.

– Contents of the list depend on what is installed!

Adding Things

• The URL Structure– Objects that are added are directly addressable

with the URL http://server.com/US/Mktg/NewFolder

which is is not on a file system!– What is the significance of this?

Adding Things

• Selecting an object to add displays an initial configuration screen

• Contents of this screen vary with the kind of object being added

• Examples…– Image, SMTPHost, ZSQL Method

Cut, Copy, Paste, Rename

• Often it is useful to re-factor a Zope layout

• Facilitated by Cut, Copy, Paste and Rename

• Available from the Workspace

• Cut not applied until subsequent Paste!

Exporting Objects

• Very useful for moving Zope applications and/or data

• Click the Export link or the Export button

• .fexp file downloaded or saved to server file system

Exporting & Inporting Objects

Importing Objects

• The export file must be in import directory• Avoid import name collisions

The Control Panel

• An important Zope artifact

• Provides a through-the-web UI for database manipulation– location– pack

• Exposes the notion of a levered product

Packing the Database

• The ZODB is a logging database

• Previous versions of objects are kept around until the database is packed

• Packing reduces the size of the database, but also eliminates the opportunity to perform Undo operations

The Zope Process

• Starting, Stopping– When– Why

User Databases

• When you provide a username and password, where does Zope look?

• Every Zope Folder can contain a single database called a User Folder

User Databases

• A User Folder will use one of several available “databases” for authentication and authorization information

• Examples:– UserDB (from an RDBMS)– LDAP– IMAPUserFolder– etcUserFolder (e.g., /etc/passwd)

Authorization

• Once we find out who you are, what level of privilege do you have?

• Your roles define your privilege in the site

Managing Users

• Users are Zope objects• Information about

users can come from Zope or foreign systems

• Zope interface allows easy add/edit/delete

Customizing Authentication

• It’s possible to customize authentication (e.g., with UserDB)

• This can be extremely powerful

Hierarchical Security

• Users defined in a higher folder exist in subfolders

• Opposite is not true

• Allows safe delegation of control

• Managers of sub-folders can easily add their own users

Creating & Managing Web Content

Key Points

• Delegating management

• Centralizing content

• Content reuse

• URLs map to objects

• Dynamic everything

What Is Content?

• Zope database is organized like a file system

• Folder and contents have direct URLs

• Content can be many things:– Static content like HTML documents, images,

and movies– Dynamic content threaded discussions– External content like relational databases,

directory servers, or email servers

Zope Content Breakthrough

• Arcane SQL and object references replaced by intuitive URLs

• Familiar interface patterned after file managers

• Powerful objects make information more useful

• All content is dynamic• Framework makes content predictable

Tracing A Request

• Before managing Zope’s content, review how Zope objects live on the web

• What happens when you do:

http://US/Sales/manage_addFolder?id=Accounting

Tracing A Request

• Zope opens the /US/ folder and grabs the Sales object

• Zope opens the Sales object and grabs the manage_addFolder object

http://US/Sales/manage_addFolder?id=Accounting

http://US/Sales/manage_addFolder?id=Accounting

Tracing A Request

• manage_addFolder is a method• Zope grabs the method and finds out what it needs to be

given as arguments• Zope finds these arguments (id) and passes them in• Zope returns the results of the method

Tracing A Request

• If a URL contains an object that can’t be found, Zope handles it

• If you need to login to grab an object, Zope handles it• If a needed argument is present, Zope handles it

http://US/Sales/manage_addFolder?id=Accounting

Tracing A Request

• The Zope management screens automate all of this• Point and click forms interface

Scenario

• ZACME is forming an Accounting division in its US region. The Accounting department will be mostly similar to the others and will manage some of its content.

Creating Content

• Objects are added through methods on a folder

• Special folders (e.g. discussions) have special content that can be added

• Kinds of objects (classes) define the form used when adding

Adding Accounting

Accounting Home Page

• Go to Accounting folder

• Add and Edit a DTML Method called index_html

Accounting Home Page

What Happened?

• index_html is a special document

• Two new objects in the database

• The following URL now works:http://server.com/US/Accounting/

Reusing Content

• How can we centralize the name of the division?

• We have a Title property on the Folder…

• How can we refer to that in our pages?

Adding DTML

• Document Template Markup Language (DTML)

• Allows simple programming to be added to documents

Using title

• Edit the index_html object

What Happened?

• The index_html object was retrieved and processed by Zope

• It asked for a property called title

• A DTML Method object gets its properties from its folder

• Zope inserted the folder’s title into index_html

Properties

• Properties help reuse information across content

• Objects (folders, documents, images, etc.) can have properties assigned to them

• Managed with the simple Zope GUI

• Properties can be strings, integers, sequences, etc.

Adding A Property

• Create a property of the Accounting folder called contact_email

Use Property In Home Page

• Add <dtml-var contact_email--> to index_html

Roles and Permissions

• Security on content can be changed with a point-and-click interface

• Some kinds of content (e.g. discussions) have extra options for the special things they do (permissions)

• These operations can be limited to certain kinds of users (roles)

Centralizing Content

• Web sites are organized hierarchically– http://server.com:8080/US/Sales/

• Some information is specific to Sales

• Some is general to US or even ZACME

Acquisition

• Zope lets you “move” content up a URL to a higher folder

• Objects lower down then just ask for the content

• Zope uses acquisition to look at all the parents for the content

Example: webmaster Property

• ZACME has a webmaster that administers all sites

• Her email address is shared across all department and division pages

Add webmaster Property

• Add property to top folder

Use webmaster Property

• All objects below can now “acquire” the webmaster property

• US/Accounting/index_html just “asks” for webmaster

Use webmaster Property

How Does Acquisition Work?

• First, acquisition is very unique and thus very tough to explain

• OO is about “is-a” and “in-a” relationships

• A Ford “is-a” Car but a Door is “in-a” Ford

• Inheritance provides “is-a”

• Zope provides “in-a”

Overriding Acquisition

• What if Accounting hired its own webmaster?

• Just define a webmaster property on the Accounting folder

• Acquisition will pick the lower one up first