bring new life to legacy code with anaconda

Bring New Life to Legacy Code with AnacondaModernize Legacy Code with the Anaconda Platform

T.J. Alumbaugh

Continuum Analytics WebinarJune 8, 2016

© 2016 Continuum Analytics

2

Today’s Presenter: T.J. Alumbaugh


T.J. AlumbaughSenior Solutions Architect

Continuum Analytics

T.J. Alumbaugh is a Senior Solutions Architect at Continuum Analytics, where he helps many of the world's largest companies solve their most challenging computational problems.

Before Continuum, T.J. worked as a software engineer, researcher and computer scientist, most notably at Lawrence Livermore National Laboratory and Sandia National Laboratories.

He holds a Masters degree in Computer Science from the University of Illinois, as well as Bachelors degrees in Mathematics and Computer Science.

3

Objectives Today• Scenario 1: Tested and Validated Legacy Analytics Need to Deployed to Windos & OS X

– Make legacy code available to your entire team– Create conda packages from Fortran code and distribute across multiple platforms

• Scenario 2: Create a “Point and Click” Interface to Legacy Code– Use your legacy code in Anaconda to create modern, interactive visualization– Create and launch a Bokeh visualization that calls Fortran code

• Scenario 3: Create performant image filters in Python & Pass to Legacy C Application– Extend the life of C code by passing python functions as a callback– Pass Numba-compiled functions as a callback to C code


DEPLOY LEGACY CODE TO WINDOWS & OS X

Scenario 1


5

Scenario 1: Deploy a Legacy Fortran Application with Anaconda

• Legacy codebases in Fortran and C/C++ exist in many large organizations

• Sometimes several hundred KLOC representing decades-worth of person hours

• Regulatory issues, cost and inertia often mean that this code is not going anywhere soon!


6


• What if developers and analysts not trained in Fortran/C/C++ could take advantage of these applications?

• What if these applications could be compiled once, and then reproducibly deployed across multiple platforms to any machine in your organization?


7


Demo: Package a Legacy Fortran application on Windows

1. Annotate the source code so that Python understands the inputs/outputs of the function

2. Build a conda package using a conda recipe that will work on multiple platforms

3. Upload the packages to a package server


8


Tools we will use:

1. Compilers (MS Visual Studio 2008 and Intel Fortran compiler)

2. F2py: tool included with NumPy that helps map Fortran functions to NumPy/Python equivalent

3. Conda-build

4. Anaconda-client


9



10


Additional Notes:

• Same process for C/C++ application, different wrapping technolgoy (SWIG, Cython)

• Conda build steps and conda recipe remain the same


CREATE A “POINT AND CLICK” INTERFACE TO LEGACY CODE

Scenario 2


12

Scenario 2: Embed Legacy Code in an Interactive Visualization with Bokeh

• Modern business tools are increasingly web-based or SaaS in nature.

• Some groups focused on your core business (and not tech) could benefit from having access to these legacy applications.


13


• What if a business analyst could have a “point and click” interface to legacy compiled code?

• What if you could easily deploy a web application within your organization to provide such an interface through a standard browser?


14


Demo: Run Fortran code inside a web application1. Install bokeh in your conda environment2. Create script with UI elements and one

callback function to call Fortran code from conda package

3. Serve bokeh application with bokeh


15


Tools we will use:

1. conda

2. Bokeh


16


Demo:


17


And Bokeh works on mobile devices as well!


18

Scenario 2: Embed Legacy Code in an Interactive Visualization with BokehAdditional Notes:

- Some amount of network configuration is typically needed to allow others to access the web application running locally on your machine.

- `bokeh serve` is great for a quick way to deploy a “POC” web application. For information on a “real” web deployment with bokeh, see the bokeh docs.


http://bokeh.pydata.org/en/latest/

http://bokeh.pydata.org/en/latest/

PASS PYTHON CODE AS A C CALLBACK

Scenario 3


20

Scenario 3: Pass Python Code as a C Callback

• Often, C/C++ libraries have APIs that take “callbacks” (functions that are called internal to the library)

• The legacy libraries have great utility, but it doesn’t make sense to continue to develop in the legacy language

• Writing “safe” code in C/C++ requires expertise that is often hard to find

© 2016 Continuum Analytics- Confidential & Proprietary

21

Scenario 3: Pass Python Code as a C Callback • What if you could leverage existing

NumPy/Python expertise in your organization to create code assets in Python?

• What if you could stop writing more code in the old language, but not sacrifice speed?


22


Demo: Create a Numba-compiled C-callback and pass it to a function in an existing C image processing library1. Compile C code as a shared library2. Load shared library into Python3. Compile a Python function with a Numba decorator4. Call C function and pass the Numba function as an

argument to the C function


23


Tools we will use:

1. Compiler: gcc on Mac OS X

2. numba

3. ctypes (comes with Python)


24


Gaussian Filter in C and in Python - Common image processing technique - Replace pixel value with weighted sum of neighboring pixels - Reduces noise in an image


25


Demo: Implement Gaussian Filter in Python, Call in C


26


Additional Notes:

- Specify argument types and result type for each function you call through the “ctypes” technique.

- Set the `errcheck` property on a foreign function. This can be used to raise an exception on return of the foreign function if the result is not as expected. See the ctypes documentation for more details.


SUMMARYWhat We Learned Today


28

Summary: What We Learned Today• Created conda packages from Fortran code and uploaded to

Anaconda Cloud

• Created and launched an interactive Bokeh visualization that calls Fortran signal processing code

• Passed a Numba-compiled function as a callback to a C library to do Gaussian filtering in Python


LEARN MORE & GO FURTHEREngage with Continuum


30

Share conda Packages Behind the Firewall


You can share and manage conda packages behind your firewall with the Repository server in Anaconda Enterprise and Anaconda Workgroup.

Contact us at [email protected] for more information, or visit https://www.continuum.io/anaconda-subscriptions.

mailto:[email protected]

https://www.continuum.io/anaconda-subscriptions



31

Learn More About conda & Numba

You can learn more about conda, Numba and Continuum Analytics’ other open source projects here:• http://conda.pydata.org/docs/• http://numba.pydata.org/


http://conda.pydata.org/docs/

http://conda.pydata.org/docs/

http://numba.pydata.org/

http://numba.pydata.org/

32

Engage with Continuum’s Team

You can hire Continuum’s team of solution architects, consultants, and Python experts to integrate your legacy applications into a modern analytics platform.

Contact us at [email protected] for more information




bring new life to legacy code with anaconda

Data & Analytics