getting started with python on ibm i. - gateway400.org

Getting started with Python on IBM i.

Install Open Source on i (LPO - 5733OPS)

• 5733-OPS has 15 options available NOW!

• The options themselves are placeholders

• Function will be delivered via PTF

5733-OPS: Install Open Source a bit unconventional?

New license program option

Open Source offerings on IBM I

15 option place holders …

… new function provided by PTF(s)

5733OPS

Program Option Description

5733OPS *BASE IBM i Open Source Solutions

5733OPS 1 IBM i Open Source Solutions Option 1 (Node.js)

5733OPS 2 IBM i Open Source Solutions Option 2 (Python)

5733OPS 3 IBM i Open Source Solutions Option 3 (GCC scripts)

5733OPS 4 IBM i Open Source Solutions Option 4



https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/IBM%20i%20Technology%20Updates/page/Open%20Source%20Technologies

Wiki page – LPP 5733OPS

Install Python on i

1. Install 5733OPS *BASE (wiki main page)

2. Install XMLSERVICE (wiki HTTP group PTFs)

3. Install 5733OPS Option 2 (wiki link python)

4. Install latest SC1 PTF's (wiki SC1 PTFs)

5. Install new-function PTF's (wiki python PTFs)

6. Verify the install by running 'python3 --version' from a terminal

Summary installing Python on ihttps://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/IBM%20i%20Technology%20Updates/page/Open%20Source%20Technologies

> call qp2term

(if green is your scene … man)

Wiki page – obtain 5733OPS

Wiki main page - left side navigation

Latest HTTP PTF Group:PTF SF99368 - level 31 for i 7.1PTF SF99713 - level 5 for i 7.2(QXMLSERV/XMLSERVICE – IBM PTF lib)

See wiki instructions

Optional – HTTP PTF group include XMLSERVICE(XMLSERVICE/XMLSERVICE – web test lib)

Wiki main page – 5733OPS

Wiki - left side navigation

python3 / python3.4 This is the main Python executable.pip3 / pip3.4 Preferred package installer. 2to3 / 2to3-3.4 Transforms Python 2.x code to Python 3.xpydoc3 / pydoc3.4 Documentation generatorpyvenv / pyvenv-3.4 Provides support for lightweight virtual environments.easy_install3 / easy_install-3.4 Lets you build, install, and manage Python packages. This command is used for installing shipped add-ons.

Base Python runtime

Wiki main page - left side navigation

DB2 connector (SI57253): easy_install3 /QOpenSys/QIBM/ProdData/OPS/Python-pkgs/ibm_db/ibm_db-*.eggToolkit for IBM i (SI57254): easy_install3 /QOpenSys/QIBM/ProdData/OPS/Python-pkgs/itoolkit/itoolkit-*3.4.egg

FastCGI gateway support (SI57255): easy_install3 /QOpenSys/QIBM/ProdData/OPS/Python-pkgs/flipflop/flipflop-*.egg

lightweight web framework (SI57256): easy_install3 /QOpenSys/QIBM/ProdData/OPS/Python-pkgs/bottle/bottle-*.egg

Installing shipped add-ons (install python PTFs first)

• Provided by the ibm_db open source project

• Incredibly robust. Can do many DB2 functions!

• Conforms to Python Database API Specification v2.0

• Complete documentation can be found here:https://code.google.com/p/ibm-db/wiki/APIs

Native DB2 connector

QSQSRVR DB2 BCI … task 1, then task 2

Simple select statement

Creating HTML table from a select

Built in DB2 Services – Use SQL to get info from the system!

Built in DB2 Services – Examples

SELECT * FROM QSYS2.USER_INFO WHERE SIGN_ON_ATTEMPTS_NOT_VALID > 0

SELECT ASP_NUMBER,UNITNBR,PERCENT_USED FROM QSYS2.SYSDISKSTAT

• Three different ways to use it:• Direct calls (like CLP)

• REST calls (local/remote)

• DB2 calls (local/remote)

Toolkit for IBM i

http://youngiprofessionals.com/wiki/index.php/XMLSERVICE/Python

http://youngiprofessionals.com/wiki/index.php/XMLSERVICE/Python

Default: DG1 ships XMLSERVICE PTFs, QXMLSERV library (PTF SI57953 - LPP OPS PTF)Optional: XMSLERVICE library installed, see following link installation http://yips.idevcloud.com/wiki/index.php/XMLService/XMLSERVICE

Transports: 1) XMLSERVICE direct call (current job) from itoolkit.lib.ilibcall import * itransport = iLibCall()

2) XMLSERVICE db2 call (QSQSRVR job) from itoolkit.db2.idb2call import * itransport = iDB2Call(config.user,config.password) -- or -- conn = ibm_db.connect(database, user, password) itransport = iDB2Call(conn)

3) XMLSERVICE http/rest/web call (Apache job) from itoolkit.rest.irestcall import * itransport = iRestCall(url, user, password)

Python egg toolkit config.py

Class list

XMLSERVICE transport - iLibCall iDB2Call iRestCallXMLSERVICE main - iToolKitXMLSERVICE call base - iBaseXMLSERVICE call *CMD or PASE - iCmd iSh iCmd5250XMLSERVICE call *PGM or *SRVPGM - iPgm iSrvPgm iParm iRet iDS iDataXMLSERVICE call DB2 - iSqlQuery iSqlPrepare iSqlExecute iSqlFetch iSqlParm iSqlFreeXMLSERVICE generic - iXml

Python egg toolkit classes

Toolkit example: rtvjoba

Toolkit example: DB2

Toolkit example: *PGM

System API

# Retrieve Hardware Resource List (QGYRHRL, QgyRtvHdwRscList) API# Service Program: QGYRHR# Default Public Authority: *USE# Threadsafe: No# Required Parameter Group:# Output Char(*)..............Receiver variable (RHRL0100, RHRL0110)# Input Binary(4).............Length of receiver variable# Input Char(8)...............Format name# Input Binary(4).............Resource category# I/O Char(*).................Error code# RHRL0100 Format# BINARY(4)...................Bytes returned# BINARY(4)...................Bytes available# BINARY(4)...................Number of resources returned# BINARY(4)...................Length of resource entry# CHAR(*).....................Resource entries# These fields repeat for each resource.# BINARY(4)...................Resource category# BINARY(4)...................Family level# BINARY(4)...................Line type# CHAR(10)....................Resource name# CHAR(4).....................Type number# CHAR(3).....................Model number# CHAR(1).....................Status# CHAR(8).....................System to which adapter is connected# CHAR(12)....................Adapter address# CHAR(50)....................Description# CHAR(24)....................Resource (liar, liar, pants on fire, binary)

System API (input)

System API (report)

System API (run)

bash-4.3$ python isrvpgm_qgyrhrl.py +++ success QGYRHR QgyRtvHdwRscList Length of receiver variable......123892 Format name......................RHRL0100 Resource category................3 RHRL0100_t: Bytes returned.................388 Bytes available................388 Number of resources returned...3 Length of resource entry.......124 -------------------------------------------------------- Resource category............3 Family level.................1 Line type....................-1 Resource name................CMB02 Type number..................268C Model number.................001 Status.......................1 System adapter connected.....*NONE Adapter address..............*NONE Description..................Comm Processor Resource kind................0000000000000001000000000000000740 -------------------------------------------------------- Resource category............3 Family level.................2 Line type....................-1 Resource name................CTL01 Type number..................6A59 Model number.................002 Status.......................1 System adapter connected.....*NONE Adapter address..............*NONE Description..................Virtual Controller Resource kind................000000000000000200000000000000010000080008 -------------------------------------------------------- Resource category............3 Family level.................3 Line type....................-1 Resource name................DSP001 : … so on ...

• We ship a fastCGI gateway, built from the flipflop open source project

FastCGI gateway

• We ship a slightly-modified version of the open-source bottle.py

• Lots of reference at http://bottlepy.org

• We'll see a small example later

Lightweight web framework

Install community packages

Usage: pip3 <command> [options]

Commands: install Install packages. uninstall Uninstall packages. freeze Output installed packages in requirements format. list List installed packages. show Show information about installed packages. search Search PyPI for packages. wheel Build wheels from your requirements. zip DEPRECATED. Zip individual packages. unzip DEPRECATED. Unzip individual packages. bundle DEPRECATED. Create pybundles. help Show help for commands.

General Options: -h, --help Show help. -v, --verbose Give more output. Option is additive, and can be used up to 3 times. -V, --version Show version and exit. -q, --quiet Give less output. --log-file <path> Path to a verbose non-appending log, that only logs failures. This log is active by default at /home/JGORZINS/.pip/pip.log. --log <path> Path to a verbose appending log. This log is inactive by default. --proxy <proxy> Specify a proxy in the form [user:passwd@]proxy.server:port. --timeout <sec> Set the socket timeout (default 15 seconds). --exists-action <action> Default action when a path already exists: (s)witch, (i)gnore, (w)ipe, (b)ackup. --cert <path> Path to alternate CA bundle.

Using the pip3 command (install from Internet)

Installing xlsxwriter

xlsxwriter example

• Most packages from the web are pure python packages. No compile needed!!

• Some packages require bits of C/C++ code

• pip3 will automatically download the source, compile and install. - By default, it will use the XL C compiler

• You will likely need to install the gcc compiler and configure according to these steps:https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/IBM%20i%20Technology%20Updates/page/Downloading%20packages%20using%20the%20pip3%20command

pip3 and native compiles

• Helpful hint!!!

Set the environment variable MAX_CONCURRENCY=1 to help debug any possible compilation problems.

pip3 and native compiles

Hello, world!

• Interactive:

• Script:

Python interactive

Let's go to the web!

fastCGI (already saw this)

http.server (included with Python)

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/IBM%20i%20Technology%20Updates/page/Sample%20web%20application%20with%20Python

bottle.py (shipped add-on)

• Quick, easy-to-use, lightweight web framework!• Lets you serve up static files, images, dynamically-generate code, and

much more

bottle.py

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/IBM%20i%20Technology%20Updates/page/Sample%20web%20application%20with%20Python

bottle (shipped add-on)

code for DB2 query function

Code for CL command function

• "The web framework for perfectionists with deadlines"

• Very powerful. Has MVC frameworks built-in. Requires a database connection for keeping relationship mappings, structural data, etc

• On IBM i, one can use MySQL via Zend Dbihttps://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/IBM%20i%20Technology%20Updates/page/First%20Django%20web%20application

Django example

http://lightbird.net/dbe2/questionnaire.html#questionnaire-model

Other versions of Python (2.6, 2.7)!

Example ready made for OPS GCC ...

Programmers want …1) Ranger python 2.7.52) Bobby python 2.6.83) Lefty python 3.4.2

Admin wants …1) No new LPARs ...2) No affecting PASE ...3) Test RPG/DB2 shared ...

Ranger

Bobby

Lefty

When a user profile with a "." (dot) in their home directory path connects using ssh, sftp, or scp path the "/" (root) directory for that connected session is set to be the portion of the directory path prior to the "."

ssh, sftp, scp, with chroot directory...

On 400 (admin *SECOFR)======> strTCPSVR SERVER(*SSHD)> mkdir -p /QOpenSys/ranger/home/ranger> CHGUSRPRF USRPRF(RANGER) LOCALE(*NONE) HOMEDIR(/QOpenSys/ranger/./home/ranger)

On laptop (user ranger)=========$ ssh ranger@myibmi

Setup chroot for 'ranger' (python 2.7.5) ...https://bitbucket.org/litmis/ibmichroot/src (see how_to_python-2.7.5.txt)===============================Installation (admin *SECOFR)===============================CRTUSRPRF USRPRF(RANGER) PASSWORD() USRCLS(*PGMR) TEXT('Tony Cairns')CHGUSRPRF USRPRF(RANGER) LOCALE(*NONE) HOMEDIR('/QOpenSys/ranger/./home/ranger')

-- outside chroot --GCC scripts provided by PTF SI58604$ ./chroot_setup.sh chroot_minimal.lst /QOpenSys/ranger$ ./chroot_setup.sh chroot_OPS_GCC.lst /QOpenSys/ranger$ ./chroot_setup.sh chroot_OPS_SC1.lst /QOpenSys/ranger-- additional setup if another profile (careful must use chroot) --$ chroot /QOpenSys/ranger /bin/bsh$ cd /$ chown -R ranger .$ exit-- back outside chroot --

Setup python 2.7 inside chroot ...https://bitbucket.org/litmis/ibmichroot/src (see how_to_python-2.7.5.txt)===============================Installation (user ranger)===============================-- inside chroot --$ ssh -X ranger@ut30p30ranger@ut30p30's password: -- if you see, use ksh --$ export PATH=/opt/freeware/bin:/usr/binPATH=/opt/freeware/bin:/usr/bin: is not an identifier-- if you see, use ksh --$ ksh $ export PATH=/opt/freeware/bin:/usr/bin$ export LIBPATH=/opt/freeware/lib:/usr/lib$ cd /QOpenSys/QIBM/ProdData/OPS/GCC$ ./pkg_setup.sh pkg_perzl_bash-4.3.lst$ bashbash-4.3$ ./pkg_setup.sh pkg_perzl_utils.lstbash-4.3$ ./pkg_setup.sh pkg_perzl_python-2.7.5.lstbash-4.3$ pythonPython 2.7.5 (default, Aug 2 2013, 23:27:48) [C] on aix5Type "help", "copyright", "credits" or "license" for more information.>>> print("Hello World")Hello World>>> bash-4.3

Simple python 2.7 web server ...https://bitbucket.org/litmis/ibmichroot/src (see simple_web_server_python.py)===============================Run simple web server (user ranger)===============================-- inside chroot --$ ssh -X ranger@ut30p30ranger@ut30p30's password: $ ksh $ export PATH=/opt/freeware/bin:/usr/bin$ export LIBPATH=/opt/freeware/lib:/usr/libbash-4.3$ python -m simple_web_server_python 8080Serving at port 8080==============================$ cat simple_web_server_python.py # python -m SimpleHTTPServer# -- or using a program --# python -m simple_web_server_python 8080import sysimport SimpleHTTPServerimport SocketServer if sys.argv[1:]: port = int(sys.argv[1])else: port = 8000 Handler = SimpleHTTPServer.SimpleHTTPRequestHandlerhttpd = SocketServer.TCPServer(("", port), Handler) print "Serving at port", porthttpd.serve_forever()

python 2.7 itoolkit egg (PTF SI57953) ...https://bitbucket.org/litmis/ibmichroot/src (see simple_web_server_python.py)===============================setup itoolkit egg (user ranger)===============================$ export PATH=/opt/freeware/bin:/usr/bin$ export LIBPATH=/opt/freeware/lib:/usr/lib$ wget https://bootstrap.pypa.io/ez_setup.py --no-check-certificate$ python ez_setup.py$ cd /QOpenSys/QIBM/ProdData/OPS/Python-pkgs/itoolkit$ easy_install itoolkit-1.1-py2.7.egg$ cp /QOpenSys/opt/freeware/lib/python2.7/site-packages/itoolkit-1.1-py2.7.egg/itoolkit/sample .$ cd sample$ python ipgm_zzcall.pyNote: By design chroot loses access to /QSYS.lIB, aka, QSH style commands do not work.Using default iLibCall transport, while *PGMs, *SRVPGMS, DB2, work fine, various CMDs, CMD5250 (dsplibl), may not work. For these tests, switch to iDB2Call or iRestCall transport, to work beyond chroot (regain access to /QSYS.LIB).

#XML Toolkit http settings (iRestCall)ScriptAlias /cgi-bin/ /QSYS.LIB/QXMLSERV.LIB/<Directory /QSYS.LIB/QXMLSERV.LIB/>AllowOverride None order allow,deny allow from all SetHandler cgi-script Options +ExecCGI</Directory>#End XML Toolkit http settings#QXMLSERV (IBM), XMLSERVICE (download), ZENDSVR6 (php), POWERRUBY, etc.

Setup chroot for 'bobby' (python 2.6.8) ...https://bitbucket.org/litmis/ibmichroot/src (see how_to_python-2.6.8.txt)===============================Installation (admin *SECOFR)===============================CRTUSRPRF USRPRF(BOBBY) PASSWORD() USRCLS(*PGMR) TEXT('Tony Cairns')CHGUSRPRF USRPRF(BOBBY) LOCALE(*NONE) HOMEDIR('/QOpenSys/bobby/./home/bobby')

-- outside chroot --GCC scripts provided by PTF SI58604$ ./chroot_setup.sh chroot_minimal.lst /QOpenSys/bobby$ ./chroot_setup.sh chroot_OPS_GCC.lst /QOpenSys/bobby$ ./chroot_setup.sh chroot_OPS_SC1.lst /QOpenSys/bobby-- additional setup if another profile (careful must use chroot) --$ chroot /QOpenSys/bobby /bin/bsh$ cd /$ chown -R bobby .$ exit-- back outside chroot --

Setup python 2.6 inside chroot ...https://bitbucket.org/litmis/ibmichroot/src (see how_to_python-2.6.8.txt)===============================Installation (user bobby)===============================-- inside chroot --$ ssh -X bobby@ut30p30bobby@ut30p30's password: -- if you see, use ksh --$ export PATH=/opt/freeware/bin:/usr/binPATH=/opt/freeware/bin:/usr/bin: is not an identifier-- if you see, use ksh --$ ksh $ export PATH=/opt/freeware/bin:/usr/bin$ export LIBPATH=/opt/freeware/lib:/usr/lib$ cd /QOpenSys/QIBM/ProdData/OPS/GCC$ ./pkg_setup.sh pkg_perzl_bash-4.3.lst$ bashbash-4.3$ ./pkg_setup.sh pkg_perzl_utils.lstbash-4.3$ ./pkg_setup.sh pkg_perzl_python-2.6.8.lstbash-4.3$ pythonPython 2.6.8 (unknown, Aug 3 2013, 00:55:15) [C] on aix5Type "help", "copyright", "credits" or "license" for more information.>>> print("Hello World")Hello World>>> ^Dbash-4.3

Simple python 2.6 web server ...https://bitbucket.org/litmis/ibmichroot/src (see how_to_python-2.6.8.txt)===============================Run simple web server (user bobby)===============================-- inside chroot --$ ssh -X bobby@ut30p30bobby@ut30p30's password: $ ksh $ export PATH=/opt/freeware/bin:/usr/bin$ export LIBPATH=/opt/freeware/lib:/usr/libbash-4.3$ python -m simple_web_server_python 8080Serving at port 8080==============================$ cat simple_web_server_python.py# python -m SimpleHTTPServer# -- or using a program --# python -m simple_web_server_python 8080import sysimport SimpleHTTPServerimport SocketServer if sys.argv[1:]: port = int(sys.argv[1])else: port = 8000 Handler = SimpleHTTPServer.SimpleHTTPRequestHandlerhttpd = SocketServer.TCPServer(("", port), Handler) print "Serving at port", porthttpd.serve_forever()

python 2.6 itoolkit egg (PTF SI57953) ...

https://bitbucket.org/litmis/ibmichroot/src (see how_to_python-2.6.8.txt===============================setup itoolkit egg (user bobby)===============================$ export PATH=/opt/freeware/bin:/usr/bin$ export LIBPATH=/opt/freeware/lib:/usr/lib$ cd /QOpenSys/QIBM/ProdData/OPS/Python-pkgs/itoolkit$ easy_install itoolkit-1.1-py2.6.egg$ cp /QOpenSys/opt/freeware/lib/python2.6/site-packages/itoolkit-1.1-py2.6.egg/itoolkit/sample .$ cd sample$ python ipgm_zzcall.pyNote: By design chroot loses access to /QSYS.lIB, aka, QSH style commands do not work.Using default iLibCall transport, while *PGMs, *SRVPGMS, DB2, work fine, various CMDs, CMD5250 (dsplibl), may not work. For these tests, switch to iDB2Call or iRestCall transport, to work beyond chroot (regain access to /QSYS.LIB).


Setup chroot for 'lefty' (python 3.4.2) ...https://bitbucket.org/litmis/ibmichroot/src (see how_to_OPS_PYTHON.txt)===============================Installation (admin (*SECOFR)===============================CRTUSRPRF USRPRF(LEFTY) PASSWORD() USRCLS(*PGMR) TEXT('Tony Cairns')CHGUSRPRF USRPRF(LEFTY) LOCALE(*NONE) HOMEDIR('/QOpenSys/lefty/./home/lefty')

-- outside chroot --GCC scripts provided by PTF SI58604$ ./chroot_setup.sh chroot_minimal.lst /QOpenSys/lefty$ ./chroot_setup.sh chroot_OPS_GCC.lst /QOpenSys/lefty$ ./chroot_setup.sh chroot_OPS_SC1.lst /QOpenSys/lefty$ ./chroot_setup.sh chroot_OPS_PYTHON.lst /QOpenSys/lefty-- additional setup if another profile (careful must use chroot) --$ chroot /QOpenSys/lefty /bin/bsh$ cd /$ chown -R lefty .$ exit-- back outside chroot --

Simple python 3.4 web server ...https://bitbucket.org/litmis/ibmichroot/src (see how_to_OPS_PYTHON.txt)===============================Run simple web server===============================-- inside chroot --$ ssh -X lefty@ut30p30lefty@ut30p30's password: $ ksh $ python3 -m simple_web_server_python 8080Serving at port 8080

import sysif sys.version_info >= (3,0): # python3 -m http.server 8080 # -- or using a program -- # python3 -m simple_web_server_python 8080 import http.server import socketserverelse: # python -m SimpleHTTPServer # -- or using a program -- # python -m simple_web_server_python 8080 import SimpleHTTPServer import SocketServer if sys.argv[1:]: port = int(sys.argv[1])else: port = 8000 if sys.version_info >= (3,0): Handler = http.server.SimpleHTTPRequestHandler httpd = socketserver.TCPServer(("", port), Handler)else: Handler = SimpleHTTPServer.SimpleHTTPRequestHandler httpd = SocketServer.TCPServer(("", port), Handler) print("Serving at port", port)httpd.serve_forever()

python 3.4 itoolkit egg (PTF SI57953) ...

https://bitbucket.org/litmis/ibmichroot/src (see how_to_OPS_PYTHON.txt)===============================setup itoolkit egg===============================$ cd /QOpenSys/QIBM/ProdData/OPS/Python-pkgs/itoolkit$ easy_install3 itoolkit-1.1-py3.4.egg$ cp /QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages/itoolkit-1.1-py3.4.egg/itoolkit/sample .$ cd sample$ python ipgm_zzcall.pyNote: By design chroot loses access to /QSYS.lIB, aka, QSH style commands do not work.Using default iLibCall transport, while *PGMs, *SRVPGMS, DB2, work fine, various CMDs, CMD5250 (dsplibl), may not work. For these tests, switch to iDB2Call or iRestCall transport, to work beyond chroot (regain access to /QSYS.LIB).


… and they all lived happily ever after ...

No problem, but you are not going to mess up my system, so chroot you.

Ranger python 2.7.5(Python presentation)

Bobby python 2.6.8(Python presentation)

Lefty python 3.4.2(Python presentation)

Wild Bill gcc 4.6.2(GCC presentation)

...except QSH, chroot not work (ILE).

Inside chroot, you can not reach /QSYS.LIB, therefore QSH is useless,

and, so is PASE system utility.

> ssh -X wildbill@ut30p30wildbill@ut30p30's password: $ system wrksyssts$ system dsplibl$ system 'no more qsys.lib'$ qshAbort$

QSH

Questions?

getting started with python on ibm i. - gateway400.org

Documents