oda for oscar

National Computational Science

ODA for OSCAR

Neil Gorsuch

NCSA

[email protected]


Database Hierarchy

• A server has one or more named databases.• ODA uses the database named “oscar” by default.• Each database has one or more named tables:

– The oscar database has “packages”, “nodes”, …

• Each table has any number of records.• Each table has named fields:

– For the “packages” table: “name”, “class”, …– Fields can be added or deleted from a table.– Fields have data types: various size strings, integer, …– Fields can have default values.– Fields can be required to have a value in each record.– Some fields can be required to be unique within a table.

• Each record has values for some or all of the fields.


ODA Interface Requirements and Capabilities

• Database interface for OSCAR clusters

• Use mySQL as the database engine

• Hide mySQL interface details

• Hide mySQL interface versions details

• Hide differing mySQL version capabilities

• Both command line and perl interfaces

• Both detailed and simple (helpful) interfaces

• Allow command line unique abbreviations

• Allow raw mySQL command syntax


Additional ODA Requirements

• Read-only non-root access

• Cluster wide access to the database

• Common command names for cli and perl

• Common arguments for cli and perl

• Error checking on arguments

• Perl functions set un-supplied arguments to defaults

• Connect/Disconnect to database done automatically


Common API Requirement

• Commands extendable through defined shortcuts

• Shortcuts can be passed arguments

• Shortcuts can use other shortcuts recursively

• Shortcuts are stored in database

• Shortcuts can include perl fragments


CLI and Perl Common Arguments

• TABLE – table name

• FIELD – field name, has to be unique in listed tables

• TABLE.FIELD – explicitly named field of named table

• ASSIGNMENT – assigns a field a new value:FIELD~VALUE TABLE.FIELD~VALUE

• MODIFIER – narrows down records output/modifiedFIELD COMPARE VALUE

FIELD COMPARE TABLE.FIELD

TABLE.FIELD COMPARE FIELD

VALUE COMPARE FIELD

• COMPARE OPERATORS: = != > < >= <= etc.


Examples Using Command Arguments

root> oda read_records packages.class name=pfilter

included

root> oda modify_records packages.name=sis vendor~yoda

root> oda read_records packages vendor=me group

Security

root> oda read_records packages.name class class=core

switcher core

sis core

perl-Qt core

c3 core

base core


Commonly Used CLI and Perl Module Commands

• add_fields – add one or more fields to a table

• create_shortcut – create a new shortcut

• create_table – create a new table

• delete_records – delete one or more table records

• delete_shortcuts – delete one or more shortcuts

• execute_command – expand and execute a command

• execute_raw_command – execute a command

• insert_record – insert one record into a table

• modify_records – modify one or more table records

• read_records – read one or more table records


Other CLI and Perl Module Commands

• connect – connect to a database

• create_database – create a new database

• disconnect – disconnect from a database

• drop_database – destroy/delete a database

• drop_tables – destroy/delete one or more tables

• list_databases – list all database names

• list_driver_types – list all database driver types

• list_fields – list the field names in a table

• list_tables – list the tables in a database


Command Line Only ODA Commands

• dump – pretty printing of database dataroot@system ]# oda dump oscar

################################################

TABLE: oscar

installation_date: <null>

interface: eth0

oscar_server_architecture: ia32

oscar_server_distribution: redhat

oscar_server_distribution_version: 9

oscar_server_node_id: <null>

oscar_version: 2.3

################################################


oda.pm read_records

# inputs: options reference to options hash

# tables_fields reference to tables/fields list

# wheres reference to where expressions list

# results reference to list or hash for results

# hash_flag non-zero if list of hashes wanted,

# zero if list of lists wanted

# errors_ref reference to error strings list

# number_ref reference to number of records modified

# outputs: non-zero is returned if no errors

# requested fields from each result record are placed

# in a list, references to those lists are placed

# in the results list that caller passed a reference to


oda.pm modify_records

# inputs: options reference to options hash

# tables pointer to tables list

# assigns pointer to assign strings list

# wheres reference to where expressions list

# hash_flag non-zero if list of hashes wanted,

# zero if list of lists wanted

# errors_ref reference to error strings list

# number_ref reference to number of records modified

# outputs: non-zero is returned if no errors

# errors strings put in errors_ref list


Database.pm database_read_table_fields

# reads specified fields from all the records in a specified database# table into a double deep hash with the first level of keys being# taken fromt the "name" field for each record, with data in the first# level of keys being pointers to hashes one per database table record,# with the second level hashes being the requested fields with their# data being the values in the database fields.## paramaters are: table database table name# fields pointer to requested field nanes list# (if undef or empty returns all fields)# wheres pointer to where expressions# (if undef all records returned)# index_field name of index field# (if undef "name" is used)# print_err if defined and non-zero, print out# error messages


Database.pm database_execute_command

# This function executes an oda database command, parsing# the command from one or more string arguments, expanding# any database shortcuts if needed. ## inputs: command_args either a single scalar string that# includes the command/shortcut and any# arguments, or a reference to a list# strings that include the command/shortcut# and any arguments.# results_ref reference to a variable for the results,# commands that return one or more strings# will place a reference to the list of# result strings in results_ref, commands# that do not return any result strings# will place the integer number of records# affected or modified in results_ref# print_errors if defined and non-zero, print out# error messages


Simple and Derived Database Tables

• Table nodes fields:– id

– name

– …

• Table node_groups fields:– id

– name

• Table node_groups_included_nodes fields:– node_id

– node_group_id


Defining Tables and Fields in config.xml

<database_tables>

<node_groups>

<fields>

<id>

<type> integer </type>

<parameters> auto_increment not null unique primary key </parameters>

</id>

<name>

<parameters> not null unique </parameters>

</name>

</fields>

</node_groups>

</database_tables>


Defining Shortcuts in Package config.xml Files

<shortcut> delete_node_group remove_all_nodes_from_node_group %1 ; delete_records node_groups name=%1</shortcut>

<shortcut> remove_all_nodes_from_node_group delete_records node_groups_included_nodes node_groups.name=%1 node_groups.id=node_groups_included_nodes.node_group_id</shortcut>


Defining Shortcuts on the Command Line

root> oda create_shortcut packages_of_class \

read_records packages.class=%1 %+

root> oda packages_of_class core

switcher

sis

perl-Qt

c3

base

root> oda packages_of_class core vendor=yoda

sis


OSCAR Simple Tables

• disk_layouts

• disk_partitions

• images

• networks

• nics

• node_groups

• nodes

• packages


OSCAR Derived Tables

• node_groups_included_nodes

• packages_filters

• packages_provides

• packages_requires

• packages_rpmlists


For Further ODA Information

• Contact me at:[email protected]

• OSCAR sourceforge project page:– http://sourceforge.net/projects/oscar/

oda for oscar

Documents