sunpy documentation - read the docs · the sunpy project is an effort to create an open-source...

SunPy DocumentationRelease 0.3.2

The SunPy Community

December 11, 2013

Contents

I About 1

1 The SunPy Team 51.1 Core Team . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.2 Developers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.3 SunPy Interns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

II SunPy User Guide 7

2 Installation 112.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.2 Installing SunPy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.3 Building from source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3 A brief tour of SunPy 213.1 1. Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.2 2. Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.3 3. Solar Physical Constants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233.4 4. Spectra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243.5 5. Lightcurves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243.6 6. Working with Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.7 7. Getting at Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.8 Querying the VSO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.9 Database Package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.10 8. Querying Helioviewer.org . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4 Acquiring Data with SunPy 294.1 Getting data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294.2 Using SunPy’s HEK module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324.3 Querying Helioviewer.org with SunPy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364.4 Using the database package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394.5 Opening Files with SunPy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

5 Data Types in SunPy 53

i

5.1 Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535.2 Lightcurves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565.3 Spectra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

6 Plotting in SunPy 636.1 1. Matplotlib Tutorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 636.2 2. Pyplot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 636.3 3. Advanced Pyplot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 646.4 4. SunPy Plotting Standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 646.5 5. peek() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 646.6 6. plot() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 656.7 7. Specifying a Colormap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

7 Time in SunPy 697.1 1. Parsing Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 697.2 2. Time Ranges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

8 Region of Interest 718.1 1. Creating an ROI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

9 Customizing sunpy 739.1 The sunpyrc file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739.2 Dynamic settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

10 Troubleshooting 7510.1 Crotate Warning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7510.2 Obtaining sunpy version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7510.3 System Info . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7610.4 sunpy install location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7610.5 .sunpy directory location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7610.6 Report a problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

III Acknowledging SunPy 79

11 Publications 83

12 Posters/Presentations 85

IV API Reference 87

13 SunPy 9113.1 sunpy Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

14 SunPy cm 9314.1 sunpy.cm Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

15 SunPy coords 9515.1 sunpy.coords Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

16 Database 9716.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9716.2 Package documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9716.3 Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

ii

17 SunPy image 10717.1 sunpy.image Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

18 SunPy Map 10918.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10918.2 Creating Map Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10918.3 Map Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11018.4 Writing a new Map Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

V Reporting Bugs 125

VI Developer’s Guide 129

19 Developer’s Guide Overview 131

20 Version Control 13320.1 Creating Your Own Repo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13320.2 Branches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13520.3 Example Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

21 Coding Standards 137

22 Documentation 13922.1 Sphinx . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13922.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14022.3 Trouble-shooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

23 Testing 14523.1 Writing a unit test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14523.2 Running unit tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14523.3 When to write unit tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14623.4 Continuous Intergration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

VII SSWIDL/SunPy Cheat Sheet 149

Python Module Index 153

Python Module Index 155

iii

Part I

About

1

SunPy Documentation, Release 0.3.2

From the outer parts of space, SunPy will soon invade Planet Earth to overthrow the horrific and honourless overloads(IDL and SolarSoft). We call upon any responsible humans who understands the pain they are in from using antiquatedlanguage and terrible software to join us! Together, with your help (either code to our repo or just being a user ofSunPy) you can make tomorrow a better tomorrow.

While she may look thin and missing parts, SunPy is very much usable. Soon, she will capable of FTL travel. Soon,ND arrays will be ours! Powering our mighty war machines, ready to crush any opposition. Come my brothers andsisters, into the breach of the IDL binary wall. It will soon crumble.

The SunPy project is an effort to create an open-source software library for solar physics using the Python program-ming language. Currently the project is in the early stages of development. If you like to get involved with the effortconsidering joining the SunPy mailing list or downloading the latest source-code from GitHub.

3

http://www.python.org/

https://groups.google.com/forum/#!forum/sunpy

https://github.com/sunpy/sunpy


4

CHAPTER 1

The SunPy Team

SunPy is a living code base with many contributors. Anyone can and is welcome to get involved.

1.1 Core Team

The core team is composed of the following members (in alphabetical order)

• Steven Christe

• Keith Hughitt

• Jack Ireland

• Stuart Mumford

• David Perez-Suarez

• Albert Shih

• Florian Mayer

1.2 Developers

The following is a list of contributors to the main SunPy packages with at least ten commits.

• Nicky Chorley

• Matt Earnshaw

• John Evans

• Nabil Freij

• Russel Hewett

• Andrew Inglis

• Simon Liedtke

• Michael Malocha

If you think you belong on this list please send an email the mailing list or make a pull request and add yourself!

5



1.3 SunPy Interns

SunPy has been partially funded by the ESA Summer of Code In Space (SOCIS) as well as the Google Summer OfCode (GSOC) programs. The following is a list of student interns along with their year of participation

• Florian Mayer (ESA Summer of Code 2011)

• Matthew Earnshaw (ESA Summer of Code 2012)

• Simon Liedtke (Google Summer of Code 2013)

• Michael Malocha (Google Summer of Code 2013)

• Thomas Meszaros (ESA Summer of Code 2013)

6 Chapter 1. The SunPy Team

Part II

SunPy User Guide

7


Welcome to the SunPy User Guide!

9


10

CHAPTER 2

Installation

2.1 Requirements

SunPy consists of many submodules that each have their own requirements. You do not need to fufill all the require-ments if you only intend on using parts of SunPy. It is however strongly recommended to have all the dependanciesinstalled (with the potential exception of glymur).

SunPy has the following strict requirements:

• Python 2.6 or 2.7

• NumPy 1.6.0 or later

• SciPy 0.10.0 or later

• AstroPy 0.3.0 or later

SunPy also depends on other packages for optional features. However, note that these only need to be installed if thoseparticular features are needed. SunPy will import even if these dependencies are not installed.

• Matplotlib [Highly Recommended]: For lightcurve, map, spectra, instr and vizualisation.

• pandas: For lightcurve.

• suds: For net.

• beautifulsoup4: For net

• glymur: To enable reading of JPEG2000 files. Glymur requires the installation of the OpenJPEG C library.Which can be found here.

• pytest: To run our tests.

2.2 Installing SunPy

These instructions assume you have a scientific Python distribution installed. If you are new to Python then you willneed to install such a distribution before continuing with these instructions.

The simplest method of obtaining a scientific Python distribution is to install the cross platform Anaconda pythondistribution. Alternatively, can follow the various platform specific instructions below:

11

http://www.python.org/

http://www.numpy.org/

http://www.scipy.org/

http://www.astropy.org/

http://http://matplotlib.org/

http://docs.python.org/library/functions.html#map

http://pandas.pydata.org/

https://bitbucket.org/jurko/suds

http://www.crummy.com/software/BeautifulSoup/

https://glymur.readthedocs.org/en/latest/

http://pytest.org/latest/


2.2.1 Windows

Overview

There are many ways to get the necessary environment up and running for SunPy on Windows, and we describe threemethods below.

Anaconda

Anaconda is an alternative cross platform Python distribution.

Anaconda is avalible from Continuum Analytics. Download the windows installer sutible for your platform and followthe install wizard.

Once complete follow the install instructions for the Anaconda python distribution.

Python(x,y)

1. Install Python(x,y)

Download and install Python(x,y). Python(x,y) is a distribution that include not only Python, but also a large varietyof Python modules and development tools. Please note that this installer is rather large (~600 MB) and thus may takea while to download. The modules in Python(x,y) include:

• NumPy

• SciPy

• Matplotlib

• PyFITS

• pandas

• distribute

• pip

• Qt

• Sphinx

• pylint

• ipython

2. Set up the Python build environment

Create a file in C:\Python27\lib\distutils\ called distutils.cfg that contains the following lines:

[build]compiler=mingw32[build_ext]compiler=mingw32

You are now ready to continue with the rest of the installation.

12 Chapter 2. Installation

http://continuum.io/downloads

https://code.google.com/p/pythonxy/wiki/Downloads


Alternate method

If Python(x,y) is unavailable, or if it is critical for you to have the most up-to-date version of modules, then thisalternate method eschews installers as much as possible and emphasizes building from source. This method is not forthe faint of heart! Please use this method only if you are experienced with computers and Python.

1. Install Python

Download and install Python 2.7 (32-bit). Note that even if you are on a 64-bit system, you are installing a 32-bitversion of Python to be able to use precompiled binaries where still needed.

You should update the PATH environment variable so that Python executables and associated scripts can be found:

1. Go to Start-> Control Panel -> System -> Advanced system settings -> Environment variables

2. Find the PATH environment variable, under either user or system variables, and append C:\Python27 andC:\Python27\Scripts, separated by semicolons.

2. Install compilers

Download and install MinGW, specifically the C, C++, and Fortran compilers. Make sure that the binaries can befound in your path (e.g., by adding C:\MinGW\bin to your path).

3. Set up the Python build environment

Create a file in C:\Python27\lib\distutils\ called distutils.cfg that contains the following lines:

[build]compiler=mingw32[build_ext]compiler=mingw32

There is currently a bug in the Python 2.7 code, so you will also need to edit cygwincompiler.py in the same directory.Remove all five instances of the character string “-mno-cygwin”.

4. Install pip

Download distribute_setup.py to be able to install distribute:

python distribute_setup.py

Download get-pip.py to be able to install pip:

python get-pip.py

5. Install required modules

You can use pip to download and build modules from source:

pip install numpypip install scipypip install matplotlib

Unfortunately, the compilations of SciPy and Matplotlib will likely fail due to missing libraries. Until there is aworkable solution, you should download the latest installers: SciPy and Matplotlib. You are now ready to continuewith the rest of the installation.

2.2. Installing SunPy 13

http://www.python.org/ftp/python/2.7.5/python-2.7.5.msi

http://mingw.org/

http://python-distribute.org/distribute_setup.py

https://raw.github.com/pypa/pip/master/contrib/get-pip.py

http://sourceforge.net/projects/scipy/files/scipy/0.12.0/scipy-0.12.0-win32-superpack-python2.7.exe/download

http://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-1.2.1/matplotlib-1.2.1.win32-py2.7.exe/download


2.2.2 Mac OS X

Overview

Because of the wide variety of methods for setting up a Python environment on Mac OS X, users have a number ofoptions with how to install SunPy and its dependencies. We have a number of solutions listed below. Choose thesolution which best suits you.

Install Anaconda

To install the Anaconda Python distribution follow the instructions here.

Warning: You must have the XCode developer package installed to be able to compile SunPy

Once complete follow the install instructions for the Anaconda python distribution.

Other installation methods

For users who wish to have more control over the installation of Python, several alternative installation methods areprovided below, including instructions for Macports and Homebrew. The following instructions are not recommendedfor beginners. OS X comes pre-loaded with Python but each versions of OS X (Mountain Lion, Snow Leopard, etc.)ships with a different version of Python. In the instructions below, we therefore recommend that you install yourown version of Python. You will then have two versions of Python living on your system at the same time. It can beconfusing to make sure that when you install packages they are installed for the correct Python so beware.

Python 2.7.3+

Download and install the latest version of 32 bit Python 2.7.3 using their DMG installer. Next, choose yourpackage installer of choice (either Macports or Homebrew) and follow the instructions below. If you do not haveeither go to their respective websites and install one of the other as needed.

Compiler

Download and install XCode from the App Store. Install the compiler by installing the optional “Command LineTools” as part of XCode. In the latest version of Xcode, this can be found in the preferences under the Downloads tab.

Macports

Macports is a useful To install everything that you need using Macports. First install Macports. You’ll need Xcodeand the command line tools. First update the package index:

sudo port selfupdate

To install the basic Python packages, run:

sudo port install py27-matplotlib py27-numpy py27-scipy

To install Qt (this will take a while!), run:


http://docs.continuum.io/anaconda/install.html

http://www.macports.org/

http://mxcl.github.com/homebrew/


sudo port install qt4-mac

This will install python 2.7 at the same time. To make the macports version of Python the default type the followinginto a terminal:

sudo port select --set python python27

For more information on using MacPorts to manage your Python installation, see the following page.

Homebrew

Homebrew is a tool for helping to automate the installation of a number of useful tools and libraries on Mac OS X. Itis similar to Macports, but attempts to improve on some of the pitfalls of Macports.

Note that if you have already installed either fink or Macports on your system, it is recommended that you uninstallthem before using Homebrew.

Next, install and update homebrew:

/usr/bin/ruby -e "$(/usr/bin/curl -fsSL https://raw.github.com/mxcl/homebrew/master/Library/Contributions/install_homebrew.rb)"brew doctor

Using homebrew, install Qt and some of the other dependencies needed for compilation later on by pip:

brew -v install gfortran pkgconfig git openjpeg readline pyqt

Now on to the next steps.

Git

Head over and download and install git.

Pip

Most Python distributions ship with a tool called easy_install which assists with installing Python packages.

Although easy_install is capable of installing most of the dependencies needed for SunPy itself, a more powerful toolcalled pip which provides a more flexible installation (including support for uninstalling, upgrading, and installingfrom remote sources such as GitHub) and should be used instead.

Use easy_install to install pip:

sudo easy_install pip

You are now ready to install scipy, numpy, and matplotlib.

Scientific Libraries

If pip installed properly, then you can install NumPy simply with:

pip install numpy

Now under Lion, install the stable version of SciPy (0.10) by running:


http://astrofrog.github.com/macports-python/

http://mxcl.github.com/homebrew/

http://pypi.python.org/pypi/setuptools


http://pypi.python.org/pypi/pip



pip install scipy

Mountain Lion users will need to install the development version of SciPy (0.11) by executing the following line:

pip install -e git+https://github.com/scipy/scipy#egg=scipy-dev

Now on to matplotlib

On Lion, install matplotlib like any other package:

pip install matplotlib

Mountain Lion users will have to use the development version as of this writing:

pip install git+https://github.com/matplotlib/matplotlib.git#egg=matplotlib-dev

Done! You are now ready to install SunPy itself .

Installation using Virtual Box

As a last resort you can always install Linux in a Virtual Machine. Virtual box is a free virtual environment thatallows you to run linux or other operating systems concurrently with OS X. Since it is very easy to install and maintainSunPy under linux, the idea here is to have your own linux environment on your mac and use it for SunPy. Werecommend you use Ubuntu linux in your virtual box. You can download an install disk for Ubuntu at their website.Directions on how to install your first virtual machine on Virtual Box are also available. After your have Ubuntuinstalled just follow the instructions for linux! Simple.

2.2.3 Linux

Overview

Almost all versions of Linux ship with a recent enough version of Python, so it is unlikely that you will need to installPython yourself.

Ubuntu

On Ubuntu, most of the pre-reqs are available in the Ubuntu software repos and can be installed using apt-get:

sudo apt-get install python-qt4sudo apt-get install git-coresudo apt-get python-numpysudo apt-get python-scipysudo apt-get python-matplotlibsudo apt-get update

Now we shall install pip.

Pip

Most Python distributions ship with a tool called easy_install which assists with installing Python packages.

Although easy_install is capable of installing most of the dependencies needed for SunPy itself, a more powerfultool called pip provides a more flexible installation (including support for uninstalling, upgrading, and installing fromremote sources such as GitHub) and should be used instead.




http://pypi.python.org/pypi/pip


Use easy_install to install pip:

sudo easy_install pip

You are now ready to install SunPy and its dependencies.

2.2.4 Using pip

To install SunPy with pip, simply run:

pip install --no-deps sunpy

Warning: Users of the Anaconda python distribution should follow the instructions for Anaconda python distri-bution.

Note: You will need a C compiler (e.g. gcc or clang) to be installed (see Building from source below) for theinstallation to succeed.

Note: If you get a PermissionError this means that you do not have the required administrative access to installnew packages to your Python installation. In this case you may consider using the --user option to install the packageinto your home directory. You can read more about how to do this in the pip documentation.

Alternatively, if you intend to do development on other software that uses SunPy, such as an affiliated package, considerinstalling SunPy into a virtualenv.

Do not install SunPy or other third-party packages using sudo unless you are fully aware of the risks.

2.2.5 Anaconda python distribution

To install the Anaconda Python distribution follow the instructions here.

Note: On OS/X you need to install XCode so you can build SunPy’s extensions.

To install SunPy launch the Anaconda command prompt or a system prompt and run the following commands.

To install SunPy’s extra dependancies run:

conda update astropyconda install suds

To install run:

conda install sunpy

To update to the latest version run:

conda update sunpy

Note: Currently Glymur / JPEG2000 support is not tested under Anaconda on any platforms. If you requireJPEG2000 support either use a different install method, or contact the SunPy mailing list.



http://www.pip-installer.org/en/1.2.1/other-tools.html#using-pip-with-the-user-scheme

http://docs.astropy.org/en/stable/development/workflow/development_workflow.html#using-virtualenv

http://docs.continuum.io/anaconda/install.html


2.2.6 Testing an installed SunPy

The easiest way to test your installed version of SunPy is running correctly is to use the sunpy.tests() function:

import sunpy.testssunpy.tests.main()

The tests should run and print out any failures, which you can report at the SunPy issue tracker.

2.3 Building from source

2.3.1 Prerequisites

You will need a compiler suite and the development headers for Python and Numpy in order to build SunPy. On Linux,using the package manager for your distribution will usually be the easiest route, while on MacOS X you will needthe XCode command line tools.

The instructions for building Numpy from source are also a good resource for setting up your environment to buildPython packages.

You will also need Cython installed to build from source, unless you are installing a numbered release. (The releasespackages have the necessary C files packaged with them, and hence do not require Cython.)

Note: If you are using MacOS X, you will need to the XCode command line tools. One way to get them is to installXCode. If you are using OS X 10.7 (Lion) or later, you must also explicitly install the command line tools. You can dothis by opening the XCode application, going to Preferences, then Downloads, and then under Components, click onthe Install button to the right of Command Line Tools. Alternatively, on 10.7 (Lion) or later, you do not need to in-stall XCode, you can download just the command line tools from https://developer.apple.com/downloads/index.action(requires an Apple developer account).

2.3.2 Obtaining the source packages

Source packages

The latest stable source package for SunPy can be downloaded here.

Development repository

The latest development version of SunPy can be cloned from github using this command:

git clone git://github.com/sunpy/sunpy.git

Note: If you wish to participate in the development of SunPy, see Developer Documentation. This document coversonly the basics necessary to install SunPy.

2.3.3 Building and Installing

SunPy uses the Python distutils framework for building and installing and requires the distribute extension–the later isautomatically downloaded when running python setup.py if it is not already provided by your system.


http://github.com/sunpy/sunpy/issues

http://docs.scipy.org/doc/numpy/user/install.html

http://cython.org/

https://developer.apple.com/xcode/

https://developer.apple.com/downloads/index.action

https://pypi.python.org/pypi/sunpy

http://docs.astropy.org/en/stable/index.html#developer-docs

http://docs.python.org/install/index.html

http://pypi.python.org/pypi/distribute


If Numpy is not already installed in your Python environment, the SunPy setup process will try to download and installit before continuing to install SunPy.

To build SunPy (from the root of the source tree):

python setup.py build

To install SunPy (from the root of the source tree):

python setup.py install

2.3.4 Troubleshooting

If you get an error mentioning that you do not have the correct permissions to install SunPy into the defaultsite-packages directory, you can try installing with:

python setup.py install --user

which will install into a default directory in your home directory.

2.3.5 Building documentation

Note: Building the documentation is in general not necessary unless you are writing new documentation or donot have internet access, because the latest (and archive) versions of SunPy’s documentation should be available atsunpy.readthedocs.org .

Building the documentation requires the SunPy source code and some additional packages:

• Sphinx (and its dependencies) 1.0 or later

• Graphviz

Note: Sphinx also requires a reasonably modern LaTeX installation to render equations. Per the Sphinx documenta-tion, for the TexLive distribution the following packages are required to be installed:

• latex-recommended

• latex-extra

• fonts-recommended

For other LaTeX distributions your mileage may vary. To build the PDF documentation using LaTeX, the fonts-extraTexLive package or the inconsolata CTAN package are also required.

There are two ways to build the SunPy documentation. The most straightforward way is to execute the command(from the sunpy source directory):

python setup.py build_sphinx

The documentation will be built in the docs/_build/html directory, and can be read by pointing a web browser todocs/_build/html/index.html.

The LaTeX documentation can be generated by using the command:

python setup.py build_sphinx -b latex

2.3. Building from source 19

http://sunpy.readthedocs.org

http://sphinx.pocoo.org

http://www.graphviz.org

http://sphinx-doc.org/builders.html?highlight=latex#sphinx.builders.latex.LaTeXBuilder

http://sphinx-doc.org/builders.html?highlight=latex#sphinx.builders.latex.LaTeXBuilder


The LaTeX file SunPy.tex will be created in the docs/_build/latex directory, and can be compiled using pdflatex.

The above method builds the API documentation from the source code. Alternatively, you can do:

cd docs/sourcemake html

And the documentation will be generated in the same location, but using the installed version of SunPy.


CHAPTER 3

A brief tour of SunPy

Welcome to the SunPy tutorial! This brief tutorial will walk you through some of the functionality currently offeredby SunPy. Start by reading this tutorial and trying out some of the examples demonstrated. Once you’ve completedthe tutorial check out the rest of the User Guide for a more thorough look at the functionality available.

3.1 1. Maps

Maps are the primary data type in SunPy they are spatially and / or temporally aware data arrays. There are types ofmaps for a 2D image, a time series of 2D images or 1D spectra or 2D spectrograms. Making a map of your data is thenormally the first step in using SunPy to work with your data.

Creating a Map

SunPy supports many different data products from various sources ‘out of the box’ we shall use SDO’s AIA instrumentas an example in this tutorial. The general way to create a map from one of the supported data products is with theMap() class from the map submodule.

Map() takes either a filename, a list of filenames or a data array and header pair. We can test map with:

import sunpyfrom sunpy.map import Mapaia = Map(sunpy.AIA_171_IMAGE)

This returns a map named aia which can be manipulated with standard SunPy map commands. For more informationabout maps checkout the map guide.

3.2 2. Plotting

Let’s begin by creating a simple plot of an AIA image. To make things easy, SunPy includes several example fileswhich are used throughout the docs. These files have names like sunpy.AIA_171_IMAGE and sunpy.RHESSI_IMAGE.

Try typing the below example into your interactive Python shell:

import sunpyfrom sunpy.map import Mapaia = Map(sunpy.AIA_171_IMAGE)aia.peek()

21

http://docs.python.org/library/functions.html#map


If everything has been configured properly you should see an AIA image with a red colormap, a colorbar on theright-hand side and a title and some labels.

There is lot going on here, but we will walk you through the example. Briefly, the first line is just importing SunPy.On the second line we create a SunPy Map object which is basically just a spatially-aware image or data array. On thelast line we then plot the map object, using the built in ‘quick plot’ function peek().

SunPy uses a matplotlib like interface to it’s plotting so more complex plots can be built by combining SunPy withmatplotlib:

import sunpyfrom sunpy.map import Mapimport matplotlib.pyplot as plt

aia = Map(sunpy.AIA_171_IMAGE)

fig = plt.figure()ax = plt.subplot(111)

aia.plot()plt.colorbar()aia.draw_limb()

plt.show()

This should output something like the image below:

22 Chapter 3. A brief tour of SunPy


3.3 3. Solar Physical Constants

SunPy contains a convenient list of solar-related physical constants. Here is a short bit of code to get you started:

from sunpy.sun import constants as con

# one astronomical unit (the average distance between the Sun and Earth)print(con.au)

# the solar radiusprint(con.radius)

Not all constants have a shortcut assigned to them (as above). The rest of the constants are stored in a dictionary. Thefollowing code grabs the dictionary and gets all of the keys.:

solar_constants = con.physical_constantssolar_constants.keys()

You can also use the following function to print out a table of all of the values available.

con.print_all()

3.3. 3. Solar Physical Constants 23


These constants are provided as a convenience so that everyone is using the same (accepted values). More will beadded over time.

3.4 4. Spectra

SunPy has spectral support for instruments which have such a capacity. CALLISTO, an international network of SolarRadio Spectrometers, is a specfic example. The main class used for this is :py:class:CallistoSpectrogram. Below is theexample built into sunpy:

from matplotlib import pyplot as pltimport sunpy

from sunpy.spectra.sources.callisto import CallistoSpectrogramimage = CallistoSpectrogram.read(sunpy.CALLISTO_IMAGE)

image.peek()

3.5 5. Lightcurves

SunPy handles time series data, fundamental to the study of any real world phenomenon, by creating a lightcurveobject. Currently lightcurve supports

• SDO/EVE

• GOES XRS

• PROBA2/LYRA

A lightcurve consits of two parts; times and measurements taken at those times. The data can either be in your currentPython session, alternatively within a local or remote file. Let’s create some fake data and pass it into a lightcurveobject:

from sunpy.lightcurve import LightCurvelight_curve = LightCurve.create({"param1": range(24*60)})



Within LightCurve.create, we have a dictionary that contains a single entry with key “param1” containing a list of1440 entries (0-1439). As there are no times provided, so a default set of times are generated.

3.6 6. Working with Times

SunPy also contains a number of convenience functions for working with dates and times. Here is a short example:

from sunpy.time import *

# parsing a standard time stringsparse_time(’2004/02/05 12:00’)

# This returns a datetime object. All SunPy functions which require# time as an input sanitize the input using parse_time.day_of_year(’2004-Jul-05 12:00:02’)

# the julian dayjulian_day((2010,4,30))

# TimeRange objects are useful for representing ranges of timetime_range = TimeRange(’2010/03/04 00:10’, ’2010/03/04 00:20’)time_range.center()

For more information about working with time in SunPy checkout the time guide.

3.7 7. Getting at Data

3.8 Querying the VSO

There are a couple different ways to query and download data from the VSO using SunPy. The method you shoulduse depends first on your preference with respect to query style: the main method of querying uses a syntax that isunique to SunPy and may require some getting used to, but is extremely flexible and powerful. To make it easy forpeople coming from SSW to get started, a second “legacy” API also exists which works is very much the same way asVSO_GET in IDL.

Further, for each of the two query APIs there are interactive and non-interactive versions available, depending on thetype of work you are doing.

The below example demonstrates a simple query for SOHO EIT data using the non-interactive version of the mainAPI:

from sunpy.net import vso

# create a new VSOClient instanceclient = vso.VSOClient()

# build our queryresult = client.query(

vso.attrs.Time((2011, 9, 20, 1), (2011, 9, 20, 2)),vso.attrs.Instrument(’eit’)

)

# print the number of matchesprint("Number of records found: %d " % result.num_records())

3.6. 6. Working with Times 25


# download matches to /download/pathres = client.get(result, path="/download/path/{file}").wait()

Note that specifying a path is optional and if you do not specify one the files will simply be downloaded into atemporary directory (e.g. /tmp/xyz). For more information about vso client checkout the vso guide.

3.9 Database Package

The database package offers the possibility to save retrieved data (e.g. via the :mod:’sunpy.net.vso’ package) onto alocal or remote database. The database may be a single file located on a local hard drive (if a SQLite database is used)or a local or remote database server. This makes it possible to fetch required data from the local database instead ofdownloading it again from a remote server.

Querying a database is straightforward, as this example using VSO, shows. The example demonstrates the usefulfeature which prevents storing the same data twice:

>>> from sunpy.database import Database>>> from sunpy.net.vso.attrs import Time, Instrument>>> db = Database(’sqlite:///’)>>> entries = db.fetch(

... Time(‘2012-08-05’, ‘2012-08-05 00:00:05’), ... Instrument(‘AIA’)) >>> assert entries is None >>> len(db) 2 >>>entries = db.fetch( ... Time(‘2012-08-05’, ‘2012-08-05 00:00:05’), ... Instrument(‘AIA’)) >>> entries is None False>>> len(entries) 2 >>> len(db) 2

Explanation: first, entries is None because the query has never been used for querying the database -> query the VSO,add new entries to database, remember query hash. In the second fetch, entries is not None because the query hasalready been used and returns a list of database entries.

3.10 8. Querying Helioviewer.org

SunPy can be used to make several basic requests using the The Helioviewer.org API including generating a PNG anddownloading a JPEG 2000 image and loading it into a SunPy Map.

A simple example of a helioviewer quiery is:

from sunpy.net.helioviewer import HelioviewerClient

hv = HelioviewerClient()hv.download_png(’2099/01/01’, 4.8, "[SDO,AIA,AIA,304,1,100]", x0=0, y0=0, width=512, height=512)

This downloads a PNG image of the latest AIA 304 image available on Helioviewer.org in the download_png com-mand 4.8 refers to the image resolution in arcseconds per pixel (larger values mean lower resolution), the “1” and“100” in the layer string refer to the visibility (visible/hidden) and opacity, x0 and y0 are the center points about whichto focus and the width and height are the pixel values for the image dimensions.

The result is:


http://helioviewer.org/api/

http://wiki.helioviewer.org/wiki/JPEG_2000


For more information checkout the helioviewer guide.

3.10. 8. Querying Helioviewer.org 27

CHAPTER 4

Acquiring Data with SunPy

4.1 Getting data

The main interface which SunPy provides to search for and download data is provided by SunPy VSO module. Thismodule provides an interface to the Virtual Solar Observatory (VSO) is a service which presents a homogenoeousinterface to heterogeneous data-sets and services. Using the VSO, a user can query multiple data providers simulta-neously, and then download the relevant data. SunPy uses the VSO through the vso module, which was developedthrough support from the European Space Agency Summer of Code in Space (ESA-SOCIS) 2011.

4.1.1 Setup

SunPy’s VSO module is in sunpy.net. It can be imported as follows:

>>> from sunpy.net import vso>>> client=vso.VSOClient()

This creates your client object. Obtaining data via the VSO is a two-stage process. You first ask the VSO to find thedata you want. The VSO queries various data-providers looking for your data. After that you choose the data youwant to download, if there is any data that matches your request. The VSO client handles the particulars of how thedata from the data provider is downloaded to your computer.

4.1.2 Searching the VSO

The vso module can be used in two different ways, an advanced query style and a simpler one based on the SSWIDL’sVSO query client, vso_search.pro (simple).

To search the VSO, you need at least a start time and end time, and an instrument. Different examples are shown inthe following sections. We will first cover the advanced style. You can skip to the simpler style below.

Advanced Query

Let’s start with a very simple query. We could ask for all SOHO/EIT data between January 1st and 2nd, 2001.

>>> qr = client.query(vso.attrs.Time(’2001/1/1’, ’2001/1/2’), vso.attrs.Instrument(’eit’))

29

http://virtualsolar.org

http://sophia.estec.esa.int/socis2011/

http://docs.virtualsolar.org/wiki/VsoIDL

http://docs.virtualsolar.org/wiki/VsoIDL

http://docs.virtualsolar.org/wiki/VsoIDL/VsoSearch


The variable qr is a Python list of response objects, each one of which is a record found by the VSO. You can findhow many records were found by typing

>>> qr.num_records()122

To get a little bit more information about the records found, try

>>> qr.show()...

Now, let’s break down the arguments of client.query to understand better what we’ve done. The first argument:

vso.attrs.Time(’2001/1/1’, ’2001/1/2’)

sets the start and end times for the query (any date/time format understood by SunPy’s parse_time function can beused to specify dates and time). The second argument:

vso.attrs.Instrument(’eit’)

sets the instrument we are looking for. So what is going on here? The notion is that a VSO query has a set of attributeobjects - described in vso.attrs - that are specifed to construct the query. For the full list of vso attributes, use

>>> help(vso.attrs)

Note that due to a current bug in the VSO, we do not recommend that the extent object vso.attrs.Extent be in yourquery. Instead, we recommend that any extent filtering you need to do be done on the queries made without settinga value to the vso.attrs.Extent object. As we will see, this query style can take more than two arguments, eachargument separated from the other by a comma. Each of those arguments are chained together using a logical AND.

This query style allows you to combine these VSO attribute objects in complex ways that are not possible with thelegacy query style.

So, let’s look for the EIT and MDI data on the same day:

>>> qr=client.query(vso.attrs.Time(’2001/1/1’, ’2001/1/2’), vso.attrs.Instrument(’eit’) | vso.attrs.Instrument(’mdi’))>>> qr.num_records()144>>> qr.show()...

The two instrument types are joined together by the operator “|”. This is the OR operator. Think of the above query assetting a set of conditions which get passed to the VSO. Let’s say you want all the EIT data from two separate days:

>>> qr=client.query(vso.attrs.Time(’2001/1/1’, ’2001/1/2’) | vso.attrs.Time(’2007/8/9’, ’2007/8/10’), vso.attrs.Instrument(’eit’) )>>> qr.num_records()227

Each of the arguments in this query style can be thought of as setting conditions that the returned records must satisfy.You can set the wavelength; for example, to return the 171 Angstrom EIT results

>>> qr=client.query(vso.attrs.Time(’2001/1/1’, ’2001/1/2’), vso.attrs.Instrument(’eit’), vso.attrs.Wave(171,171) )>>> qr.num_records()4

Simple Query

If you just need to do a quick query or don’t want to do anything too complicated you can use the simple query. Let’sshow now how the first example is executed with the legacy mode.

As before, we want EIT data between 2001/01/01 and 2001/01/02

30 Chapter 4. Acquiring Data with SunPy


>>> qr=client.query_legacy(tstart=’2001/01/01’, tend=’2001/01/02’, instrument=’EIT’)

which is almost identical to what you would type in SSWIDL. So, what’s happening with this command? The clientis going out to the web to query the VSO to ask how many files EIT images are in the archive between the start of2001/01/01 and the start of 2001/01/02. The same query can also be performed using a slightly different syntax. Forexample

>>> qr=client.query_legacy(’2001/1/1’, ’2001/1/2’, instrument=’EIT’)

both gives the same result. The variable qr is a Python list of response objects, each one of which is a record found bythe VSO. How many records have been found? You can find that out be typing

>>> qr.num_records()122

To get a little bit more information, try

>>> qr.show()...

The Solarsoft legacy query has more keywords available: to find out more about the legacy query, type:

>>> help(client.query_legacy)

As an example, let’s say you just want the EIT 171 Angstrom files for that data. These files can be found by

>>> qr=client.query_legacy(tstart=’2001/01/01’, tend=’2001/01/02’, instrument=’EIT’, min_wave=’171’, max_wave=’171’, unit_wave=’Angstrom’)

which yields four results, the same as the VSO IDL client.

4.1.3 Downloading data

All queries return a query response list. This list can then used to get the data. This list can also be edited as you seefit. For example you can further reduce the number of results and only get those. So having located the data you want,you can download it using the following command:

>>> res=client.get(qr, path=’/ThisIs/MyPath/to/Data/{file}.fits’)

This downloads the query results into the directory /ThisIs/MyPath/to/Data naming each downloaded file with thefilename {file} obtained from the VSO , and appended with the suffix .fits. The {file} option uses the file nameobtained by the VSO for each file. You can also use other properties of the query return to define the path where thedata is saved. For example, to save the data to a subdirectory named after the instrument, use

>>> res=client.get(qr, path=’/ThisIs/MyPath/to/Data/{instrument}/{file}.fits’)

If you have set your default download directory in your sunpyrc configuration file then you do not need to identify apath at all. All you data will be downloaded there.

Note that the download process is spawned in parallel to your existing Python session. This means that the remainderof your Python script will continue as the download proceeds. This may cause a problem if the remainder of yourscript relies on the presence of the downloaded data. If you want to resume your script after all the data has beendownloaded then append .wait() to the get command above, i.e.,

>>> res=client.get(qr, path=’/Users/ireland/Desktop/Data/{instrument}/{file}.fits’).wait()

More information on the options available can be found through the standard Python help command.

4.1. Getting data 31


4.2 Using SunPy’s HEK module

The Heliophysics Event Knowledgebase (HEK) is a repository of feature and event information concerning the Sun.Entries are generated both by automated algorithms and human observers. SunPy accesses this information throughthe ‘hek’ module, which was developed through support from the European Space Agency Summer of Code in Space(ESA-SOCIS) 2011.

4.2.1 1. Setting up the client

SunPy’s HEK module is in sunpy.net. It can be imported into your IPython session as follows:

>>> from sunpy.net import hek>>> client = hek.HEKClient()

This creates a client that we will use to interact with the HEK.

4.2.2 2. A simple query

To search the HEK, you need a start time, an end time, and an event type. Times are specified as Python datetimeobjects. Event types are specified as upper case, two letter strings, and are identical to the two letter abbreviationsfound at the HEK website, http://www.lmsal.com/hek/VOEvent_Spec.html.

>>> tstart = ’2011/08/09 07:23:56’>>> tend = ’2011/08/09 12:40:29’>>> event_type = ’FL’>>> result = client.query(hek.attrs.Time(tstart,tend),hek.attrs.EventType(event_type))

The first line in the block of code above imports the datetime module. The second and third lines define the searchstart and end times. The fourth line specifies the event type, in this ‘FL’ or flare. Line 5 goes out to the web, contactsthe HEK, and queries it for the information you have requested. Event data for ALL flares available in the HEK withinthe time range 2011/08/09 07:23: 56 UT - 2011/08/09 12:40:20 UT will be returned, regardless of which featurerecognition method used to detect the flare.

Let’s break down the arguments of client.query. The first argument:

hek.attrs.Time(tstart,tend)

sets the start and end times for the query. The second argument:

hek.attrs.EventType(event_type)

sets the type of event to look for. Since we have defined event_type = ‘FL’, this sets the query to look for flares. Wecould have also set the flare event type using the syntax

hek.attrs.FL

There is more on the attributes of hek.attrs in section 4 of this guide.

4.2.3 3. The result

So, how many flare detections did the query turn up?

>>> len(result)19


http://www.lmsal.com/hek/VOEvent_Spec.html


The object returned by the above query is a list of Python dictionary objects. Each dictionary consists of key-valuepairs that exactly correspond to the parameters listed at http://www.lmsal.com/hek/VOEvent_Spec.html. You caninspect all the dictionary keys very simply:

>>> result[0].keys()[u’skel_startc1’,u’concept’,u’frm_versionnumber’,u’hrc_coord’,u’refs_orig’,....

and so on. Remember, the HEK query we made returns all the flares in the time-range stored in the HEK, regardlessof the feature recognition method. The HEK parameter which stores the the feature recognition method is called“frm_name”. Using list comprehensions (which are very cool), it is easy to get a list of the feature recognitionmethods used to find each of the flares in the result object, for example:

>>> [elem["frm_name"] for elem in result][u’asainz’,u’asainz’,u’asainz’,u’asainz’,u’asainz’,u’asainz’,u’asainz’,u’SSW Latest Events’,u’SEC standard’,u’Flare Detective - Trigger Module’,u’Flare Detective - Trigger Module’,u’SSW Latest Events’,u’SEC standard’,u’Flare Detective - Trigger Module’,u’Flare Detective - Trigger Module’,u’Flare Detective - Trigger Module’,u’Flare Detective - Trigger Module’,u’Flare Detective - Trigger Module’]

It is likely each flare on the Sun was actually detected multiple times by many different methods.

4.2.4 4. More complex queries

The HEK client allows you to make more complex queries. There are two key features you need to know in order tomake use of the full power of the HEK client. Firstly, the attribute module - hek.attrs - describes ALL the parame-ters stored by the HEK as listed in http://www.lmsal.com/hek/VOEvent_Spec.html, and the HEK client makes theseparameters searchable.

To explain this, let’s have a closer look at hek.attrs. The help command is your friend here; scroll down to sectionDATA you will see:

>>> help(hek.attrs)AR = <sunpy.net.hek.attrs.AR object>Area = <sunpy.net.hek.attrs.Area object>Bound = <sunpy.net.hek.attrs.Bound object>BoundBox = <sunpy.net.hek.attrs.BoundBox object>CC = <sunpy.net.hek.attrs.CC object>CD = <sunpy.net.hek.attrs.CD object>CE = <sunpy.net.hek.attrs.CE object>CH = <sunpy.net.hek.attrs.EventType object>CJ = <sunpy.net.hek.attrs.EventType object>

4.2. Using SunPy’s HEK module 33




CR = <sunpy.net.hek.attrs.EventType object>CW = <sunpy.net.hek.attrs.EventType object>EF = <sunpy.net.hek.attrs.EF object>ER = <sunpy.net.hek.attrs.EventType object>Event = <sunpy.net.hek.attrs.Event object>FA = <sunpy.net.hek.attrs.EventType object>FE = <sunpy.net.hek.attrs.EventType object>FI = <sunpy.net.hek.attrs.FI object>FL = <sunpy.net.hek.attrs.FL object>FRM = <sunpy.net.hek.attrs.FRM object>etc etc...

The object hek.attrs knows the attributes of the HEK. You’ll see that one of the attributes is a flare object

FL = <sunpy.net.hek.attrs.FL object>

We can replace hek.attrs.EventType(‘FL’) with hek.attrs.FL - they do the same thing, setting the query to look for flareevents. Both methods of setting the event type are provided as a convenience

Let’s look further at the FRM attribute:

>>> help(hek.attrs.FRM)Help on FRM in module sunpy.net.hek.attrs object:class FRM(__builtin__.object)| Data descriptors defined here:|| __dict__| dictionary for instance variables (if defined)|| __weakref__| list of weak references to the object (if defined)|| ----------------------------------------------------------------------| Data and other attributes defined here:|| Contact = <sunpy.net.hek.attrs._StringParamAttrWrapper object>|| HumanFlag = <sunpy.net.hek.attrs._StringParamAttrWrapper object>|| Identifier = <sunpy.net.hek.attrs._StringParamAttrWrapper object>|| Institute = <sunpy.net.hek.attrs._StringParamAttrWrapper object>|| Name = <sunpy.net.hek.attrs._StringParamAttrWrapper object>|| ParamSet = <sunpy.net.hek.attrs._StringParamAttrWrapper object>|| SpecificID = <sunpy.net.hek.attrs._StringParamAttrWrapper object>|| URL = <sunpy.net.hek.attrs._StringParamAttrWrapper object>|| VersionNumber = <sunpy.net.hek.attrs._StringParamAttrWrapper object>

Let’s say I am only interested in those flares identified by the SSW Latest Events tool. I can retrieve those entries onlyfrom the HEK with the following command:

>>> result = client.query( hek.attrs.Time(tstart,tend), hek.attrs.EventType(event_type), hek.attrs.FRM.Name == ’SSW Latest Events’)>>> len(result)2



We can also retrieve all the entries in the time range which were not made by SSW Latest Events with the followingcommand:

>>> result = client.query( hek.attrs.Time(tstart,tend), hek.attrs.EventType(event_type),hek.attrs.FRM.Name != ’SSW Latest Events’)>>> len(result)17

We are using Python’s comparison operators to filter the returns from the HEK client. Other comparisons are possible.For example, let’s say I want all the flares that have a peak flux of over 4000.0:

>>> result = client.query(hek.attrs.Time(tstart,tend), hek.attrs.EventType(event_type), hek.attrs.FL.PeakFlux > 4000.0)>>> len(result)1

Multiple comparisons can be included. For example, let’s say I want all the flares with a peak flux above 1000 ANDwest of 800 arcseconds from disk center of the Sun

>>> result = client.query(hek.attrs.Time(tstart,tend), hek.attrs.EventType(event_type), hek.attrs.Event.Coord1 > 800, hek.attrs.FL.PeakFlux > 1000.0)

Multiple comparison operators can be used to filter the results back from the HEK.

The second important feature about the HEK client is that the comparisons we’ve made above can be combined usingPython’s logical operators. This makes complex queries easy to create. However, some caution is advisable. Let’s sayI want all the flares west of 50 arcseconds OR have a peak flux over 1000.0:

>>> result = client.query(hek.attrs.Time(tstart,tend), hek.attrs.EventType(event_type), (hek.attrs.Event.Coord1 > 50) or (hek.attrs.FL.PeakFlux > 1000.0) )

and as a check

>>> [elem["fl_peakflux"] for elem in result][None,None,None,None,None,None,None,2326.86,1698.83,None,None,2360.49,3242.64,1375.93,6275.98,923.984]

>>> [elem["event_coord1"] for elem in result][51,51,51,924,924,924,69,883.2,883.2,69,69,883.2,

4.2. Using SunPy’s HEK module 35


883.2,883.2,883.2,883.2]

Note that some of the fluxes are returned as “None”. This is because some feature recognition methods for flares donot report the peak flux. However, because the location of event_coord1 is greater than 50, the entry from the HEKfor that flare detection is returned.

Let’s say we want all the flares west of 50 arcseconds AND have a peak flux over 1000.0:

>>> result = client.query(hek.attrs.Time(tstart,tend), hek.attrs.EventType(event_type), (hek.attrs.Event.Coord1 > 50) and (hek.attrs.FL.PeakFlux > 1000.0) )

>>> [elem["fl_peakflux"] for elem in result][2326.86, 1698.83, 2360.49, 3242.64, 1375.93, 6275.98]>>> [elem["event_coord1"] for elem in result][883.2, 883.2, 883.2, 883.2, 883.2, 883.2]

In this case none of the peak fluxes are returned with the value “None”. Since we are using an “and” logical operatorwe need a result from the “(hek.attrs.FL.PeakFlux > 1000.0)” filter. Flares that have “None” for a peak flux cannotprovide this, and so are excluded. The “None” type in this context effectively means “Don’t know”; in such cases theclient returns only those results from the HEK that definitely satisfy the criteria passed to it.

4.3 Querying Helioviewer.org with SunPy

SunPy can be used to make several basic requests using the The Helioviewer.org API including generating a PNG anddownloading a JPEG 2000 image and loading it into a SunPy Map.

The SunPy Helioviewer client requires installing two other pieces of software. The first OpenJPEG is an open sourcelibrary for reading and writing JPEG2000 files. To install OpenJPEG, please follow the instructions at the OpenJPEGhomepage.

The other package you will need is Glymur. Glymur is an interface between Python and the OpenJPEG linraries.Please follow the instructions here to install Glymur on your system.

To interact with the Helioviewer API, users first create a “HelioviewerClient” instance. The client instance can then beused to make various queries against the API using the same parameters one would use when making a web request.

Nearly all requests require the user to specify the data they are interested in and this can be done using one of twomethods:

1. Call “get_data_sources()” to get a list of the data that is available, and use the source id numbers referenced inthe result to refer to a particular dataset, or,

2. Specify the four components of a Helioviewer.org data source or layer: observatory, instrument, detector andmeasurement.

Let’s begin by getting a list of data sources available on the server using the get_datasources method:

from sunpy.net.helioviewer import HelioviewerClient

hv = HelioviewerClient()datasources = hv.get_data_sources()

# print a list of datasources and their associated idsfor observatory, instruments in datasources.items():

for inst, detectors in instruments.items():for det, measurements in detectors.items():



http://wiki.helioviewer.org/wiki/JPEG_2000

http://www.openjpeg.org

http://www.openjpeg.org

https://pypi.python.org/pypi/Glymur/

https://glymur.readthedocs.org/en/latest/


for meas, params in measurements.items():print("%s %s: %d" % (observatory, params[’nickname’], params[’sourceId’]))

Suppose we next want to download a PNG image of the latest AIA 304 image available on Helioviewer.org. We coulduse the explicit approach:

hv.download_png(’2099/01/01’, 4.8, "[SDO,AIA,AIA,304,1,100]", x0=0, y0=0, width=512, height=512)

Where 4.8 refers to the image resolution in arcseconds per pixel (larger values mean lower resolution), the “1” and“100” in the layer string refer to the visibility (visible/hidden) and opacity, x0 and y0 are the center points about whichto focus and the width and height are the pixel values for the image dimensions.

The result is:

Note that the filename of the returned file has the date and time of the request, not of any of the times shown in theimage itself. This is not a bug. Helioviewer.org finds images closest to the requested time. Since the user may ask for

4.3. Querying Helioviewer.org with SunPy 37


images from multiple sources, and each of them may have a different observation time, the problem becomes whichtime is the most appropriate to associate with the resultant image. Helioviewer.org doesn’t choose between the imagestimes, but instead uses the request time to construct the image filename. This means that the image file names forrequest times in the future (like in this example) can look a little unusual compared to the times in the image.

If we find that the source id for AIA 304 is is 13, we could make the same request using:

hv.download_png(’2099/01/01’, 4.8, "[13,1,100]", x0=0, y0=0, width=512, height=512)

Now suppose we wanted to create a composite PNG image using data from two different AIA wavelengths and LASCOC2 coronagraph data. The layer string is extended to include the additional data sources, and opacity is throttled downfor the second AIA layer so that it does not completely block out the lower layer:

hv.download_png(’2099/01/01’, 6, "[SDO,AIA,AIA,304,1,100],[SDO,AIA,AIA,193,1,50],[SOHO,LASCO,C2,white-light,1,100]", x0=0, y0=0, width=768, height=768)

The result looks like:



Next, let’s see how we can download a JPEG 2000 image and load it into a SunPy Map object.

The overall syntax is similar to the download_png request, expect instead of specifying a single string to indicatewhich layers to use, here we can specify the values as separate keyword arguments:

filepath = hv.download_jp2(’2012/07/05 00:30:00’, observatory=’SDO’, instrument=’HMI’, detector=’HMI’, measurement=’continuum’)from sunpy.map import Maphmi = Map(filepath)hmi.submap([200,550],[-400,-200]).peek()

For more information about using querying Helioviewer.org, see the Helioviewer.org API documentation at:http://helioviewer.org/api/.

4.4 Using the database package

The database package offers the possibility to save retrieved data (e.g. via the sunpy.net.vso package) onto a localor remote database. The database may be a single file located on a local hard drive (if a SQLite database is used) ora local or remote database server (see the SQLAlchemy documentation for a list of supported databases) This makesit possible to fetch required data from the local database instead of downloading it again from a remote server. Thepackage sunpy.database was developed as part of Google Summer of Code (GSOC) 2013.

4.4.1 1. Connecting and initializing the database

To start a new connection to an already existing or to a new database, instantiate a new Database object. The firstparameter of Database receives one mandatory argument: A URL which describes how to connect to the database.This value is directly passed to sqlalchemy.create_engine(). The supported formal of this URL is described bythe documentation of sqlalchemy.create_engine() as follows:

“The string form of the URL is dialect+driver://user:password@host/dbname[?key=value..],where dialect is a database name such as mysql, oracle, postgresql, etc., and driver the name of aDBAPI, such as psycopg2, pyodbc, cx_oracle, etc.”

4.4. Using the database package 39


http://docs.sqlalchemy.org/en/rel_0_8/dialects/

http://www.google-melange.com/gsoc/homepage/google/gsoc2013

http://docs.sqlalchemy.org/en/rel_0_8/core/engines.html#sqlalchemy.create_engine



The database object in our example connects to a new SQLite database with the file name “sunpydata.sqlite” in thecurrent directory. Note that a connection is only established when it’s really needed, i.e. if some query is sent to thedatabase to read from it. Transactions can also be committed explicitly using the Database.commit() method.

>>> from sunpy.database import Database>>> database = Database(’sqlite:///sunpydata.sqlite’)

Warning: If you are using Database objects in an interactive Python session you must not forget to call theDatabase.commit() method on them explicitly before quitting the Python session! Otherwise, all changes on thealtered databases are lost!

4.4.2 2. Adding new entries

Before explaining how to add new entries, it is important to know what information is saved in an entry. Each databaseentry is an instance of the class tables.DatabaseEntry with the following attributes:

Attribute Descriptionid A unique ID number. By default it is None, but automatically set to the maximum number plus

one when an entry is added to the database.source The source is the name of an observatory or the name of a network of observatories.provider The name of the server which provides the retrieved data.physobs A physical observable identifier used by VSO.fileid The file ID is a string defined by the data provider that should point to a specific data product. The

association of fileid to the specific data may change sometime, if the fileid always points to thelatest calibrated data.

observa-tion_time_start

The date and time when the observation of the data started.

observa-tion_time_end

The date and time when the observation of the data ended.

instrument The instrument which was used to observe the data.size The size of the data in kilobytes (-1 if unknown).wavemin The value of the measured wave length.wavemax This is the same value as wavemin. The value is stored twice, because each

suds.sudsobject.QueryResponseBlock which is used by the vso package contains both thesevalues.

path A local file path where the according FITS file is saved.down-load_time

The date and time when the files connected to a query have been downloaded. Note: this is notthe date and time when this entry has been added to a database!

starred Entries can be starred to mark them. By default, this value is False.fits_header_entriesA list of tables.FitsHeaderEntry instances.tags A list of tables.Tag instances.

The id attribute is automatically set if an entry is added to a database. The attributes source, provider, physobs,fileid, observation_time_start, observation_time_end, instrument, size, wavemin, and wavemax are setby methods which use the VSO interface. In particular, these are Database.add_from_vso_query_result(),Database.download() and possibly Database.fetch(). The attributes path and download_time are set bythe method Database.download() and also possibly by Database.fetch(). starred is set or changed via themethod Database.star() or unstar(), respectively. Analogously, tags is set via the methods Database.tag()and Database.remove_tag(). The attribute fits_header_entries is set by the methods Database.download(),Database.add_from_dir(), and Database.add_from_file().



2.1 Adding entries from one FITS file

The method Database.add_from_file() receives one positional argument (either a string or a file-like object) whichis used to add at least one new entry from the given FITS file to the database. Why “at least one” and not “exactlyone”? The reason is that each database entry does not represent one file but one FITS header of a file. That means,if you pass a file which has 5 FITS headers in it, 5 entries will be added to the database. The file in the followingexample (sunpy.AIA_171_IMAGE) has only one FITS header, that is why just one entry is added to the database.

The method saves the value of path by either simply passing on the value of the received argument (if it was a string)or by reading the value of file.name where file is the passed argument. If the path cannot be determined, it stays toNone (the default value).

The values of wavemin and wavemax are only set if the wavelength unit of the passed FITS file can be found out orif the attribute default_waveunit of the database object is set. These values are then used to convert from the usedunit to nanometers. The rationale behind this behaviour is that it makes querying for wavelengths more flexible. tl;dr:wavelengths are always stored in nanometers!

The value of the attribute instrument is simply set by looking up the FITS header key INSTRUME. The value ofobservation_time_start is set by searching for the FITS header key DATE-OBS or DATE_OBS. Analogously,observation_time_end is set by searching for DATE-END or DATE_END. Finally, the whole FITS header is storedin the attribute fits_header_entries as a list of tables.FitsHeaderEntry instances. All FITS comments arestored in the attribute fits_key_comments which is a list of tables.FitsKeyComment instances.

Using the function len on a Database object returns the number of saved database entries. To get the first entry of thedatabase, database[0] is used (the ID number of the entries does not matter, database[0] always returns the oldestsaved entry of the database). If the database had been empty, this expression would have raised an IndexError. Insection 3, more advanced formats of the slicing syntax are introduced.

>>> import sunpy>>> database.add_from_file(sunpy.AIA_171_IMAGE)>>> len(database)1>>> entry = database[0]>>> entry.path == sunpy.AIA_171_IMAGETrue>>> entry.wavemin, entry.wavemax(17.1, 17.1)>>> entry.instrument’AIA_3’>>> entry.observation_time_start, entry.observation_time_end(datetime.datetime(2011, 3, 19, 10, 54, 0, 340000), None)>>> len(entry.fits_header_entries)170>>> for fits_header_entry in entry.fits_header_entries[:10]:... print ’{entry.key}\n\t{entry.value}’.format(entry=fits_header_entry)...SIMPLE

1BITPIX

32NAXIS

2NAXIS1

1024NAXIS2

1024EXTEND

1COMMENT



FITS (Flexible Image Transport System) format is defined in ’Astronomy and Astrophysics’, volume 376, page 359; bibcode: 2001A&A...376..359HORIGIN

SDO/JSOC-SDPDATE

2011-03-19T11:08:25TELESCOP

SDO/AIA>>> for fits_key_comment in entry.fits_key_comments:... print ’{comment.key}\n\t{comment.value}’.format(comment=fits_key_comment)...NAXIS

number of data axesNAXIS1

length of data axis 1DATASUM

data unit checksum updated 2011-03-19T11:08:18EXTEND

FITS dataset may contain extensionsBITPIX

number of bits per data pixelSIMPLE

file does conform to FITS standardCHECKSUM

HDU checksum updated 2011-03-19T11:08:18NAXIS2

length of data axis 2

2.2 Adding entries from a directory of FITS files

Adding all FITS files from a certain directory works by calling the method Database.add_from_dir() and passingthe desired directory to it. By setting the keyword argument ignore_already_added to True, no exception is raisedif it is attempted to add an already existing entry (In this case, setting this parameter is required because the filesunpy.AIA_171_IMAGE was already added which is located in the directory sunpy.data.sample.rootdir).

>>> database.default_waveunit = ’angstrom’>>> database.add_from_dir(sunpy.data.sample.rootdir, ignore_already_added=True)>>> len(database)21

2.3 Adding entries using the VSO interface

2.3.1 Adding entries from a VSO query result

A VSO query result can be used to add new entries to the database. The number of database entries thatwill be added is equal to the value of qr.num_records() in the following code snippet. Note that the methodDatabase.add_from_vso_query_result() does not download any files, though. If you want to add new entriesusing the VSO and also want to download files at the same time, take a look at the following two sections.

>>> from sunpy.net import vso>>> client = vso.VSOClient()>>> qr = client.query(... vso.attrs.Time(’2011-05-08’, ’2011-05-08 00:00:05’),... vso.attrs.Instrument(’AIA’))>>> qr.num_records()4


http://docs.python.org/library/constants.html#True


>>> database.add_from_vso_query_result(qr)>>> len(database)25

2.3.2 Downloading

The method Database.download() queries the VSO by passing the given query on tosunpy.net.vso.VSOClient.query(). Note that not the number of records of the resulting query result deter-mines the number of entries that will be added to the database! The number of entries that will be added depends onthe total number of FITS headers. The download method also accepts an optional keyword argument path which ispassed as-is to sunpy.net.vso.VSOClient.get() and determines the value of the path attribute of each entry.

>>> database.download(... vso.attrs.Time(’2012-08-05’, ’2012-08-05 00:00:05’),... vso.attrs.Instrument(’AIA’))>>> len(database)27

2.3.3 “Clever” Fetching

The method Database.fetch() queries the database if the given query has already been used once to add entriesusing the method Database.download(). Otherwise, the given query is used to download and add new data viaDatabase.download(). This means: If you are not sure whether the required data already exists in the database, useDatabase.fetch() and you use a method to save bandwidth!

As you can see in the following example, the first call of the fetch method results in a list of entries. These are theentries which have been returned by querying the database. This is the reason why the number of saved entries in thedatabase is still 27 even after the fetch method has been called. The second fetch call has not been done already on adownload call, therefore the given query is used to download new data and add these resulting entries to the database.Because the query result translates to 2 records, 2 new entries are added to the database after the fetch call.

>>> entries = database.fetch(... vso.attrs.Time(’2012-08-05’, ’2012-08-05 00:00:05’),... vso.attrs.Instrument(’AIA’))>>> entries is NoneFalse>>> len(entries)2>>> len(database)27

>>> entries = database.fetch(... vso.attrs.Time(’2013-08-05’, ’2013-08-05 00:00:05’),... vso.attrs.Instrument(’AIA’))>>> entries is NoneTrue>>> len(database)29

2.4 Adding entries manually

Although usually not required, it is also possible to add database entries by specifying the parameters manually. To doso, you simply pass the values as keyword arguments to tables.DatabaseEntry as follows:



>>> from sunpy.database.tables import DatabaseEntry>>> entry = DatabaseEntry(instrument=’EIT’, wavemin=25.0)>>> database.add(entry)>>> entry in databaseFalse>>> database.commit()>>> entry in databaseTrue>>> len(database)30

Note that the in operator works only as expected after the Database.commit() method has been called!

4.4.3 3. Displaying entries in a table

Meanwhile, 30 entries have been added, all of them saving a lot of data. How can selected data of each entry bedisplayed? Fortunately, there is a helper function to do this: tables.display_entries() takes two arguments:the first one is an iterator of tables.DatabaseEntry instances. Remember that an instance of Database yieldsinstances of tables.DatabaseEntry instances, so you can simply pass a database object. The second argumentis an iterable of the resulting columns in the table to be displayed. Each string in this iterable is used to accessthe entry’s attribute values. In the following example, the values of entry.id, entry.observation_time_start,entry.observation_time_end, entry.instrument, entry.wavemin, and entry.wavemax are displayed (whereentry stands for the respective database entry). Note that N/A is displayed if the value cannot be found or is notset.

>>> from sunpy.database.tables import display_entries>>> print display_entries(... database,... [’id’, ’observation_time_start’, ’observation_time_end’, ’instrument’, ’wavemin’, ’wavemax’])id observation_time_start observation_time_end instrument wavemin wavemax-- ---------------------- -------------------- ---------- ------- -------1 2011-03-19 10:54:00.340000 N/A AIA_3 17.1 17.12 2010-10-16 19:12:18 2010-10-16 19:12:22 RHESSI N/A N/A3 N/A N/A N/A N/A N/A4 N/A N/A N/A N/A N/A5 N/A N/A N/A N/A N/A6 2002-06-25 10:00:10.514000 N/A EIT 19.5 19.57 2011-03-19 10:54:00.340000 N/A AIA_3 17.1 17.18 2011-09-22 00:00:00 2011-09-22 00:00:00 BIR N/A N/A9 N/A N/A N/A N/A N/A10 2002-02-20 11:06:00 2002-02-20 11:06:43.330000 RHESSI N/A N/A11 N/A N/A N/A N/A N/A12 N/A N/A N/A N/A N/A13 N/A N/A N/A N/A N/A14 N/A N/A N/A N/A N/A15 N/A N/A N/A N/A N/A16 N/A N/A N/A N/A N/A17 N/A N/A N/A N/A N/A18 N/A N/A N/A N/A N/A19 N/A N/A N/A N/A N/A20 N/A N/A N/A N/A N/A21 2012-01-01 00:16:07.836000 N/A SWAP 17.4 17.422 2011-05-08 00:00:00 2011-05-08 00:00:01 AIA 17.1 17.123 2011-05-08 00:00:00 2011-05-08 00:00:01 AIA 21.1 21.124 2011-05-08 00:00:02 2011-05-08 00:00:03 AIA 9.4 9.425 2011-05-08 00:00:03 2011-05-08 00:00:04 AIA 33.5 33.526 2012-08-05 00:00:01 2012-08-05 00:00:02 AIA 9.4 9.4



27 2012-08-05 00:00:02 2012-08-05 00:00:03 AIA 33.5 33.528 2013-08-05 00:00:01 2013-08-05 00:00:02 AIA 9.4 9.429 2013-08-05 00:00:02 2013-08-05 00:00:03 AIA 33.5 33.530 N/A N/A EIT 25.0 N/A

In Section 2.1, “Adding entries from one FITS file”, it has already been shows that the index operator can be usedto access certain single database entries like database[5] or even use negative indices such as in database[-2] toget the second-latest entry. It is also possible to use the more advanced slicing syntax: database[:] returns a list ofall database entries (and is hereby an alias for list(database)), database[::2] returns a list of every 2nd entry,starting with the oldest one. As you can imagine, database[9::10] starts with the 10th entry and returns a list ofevery 10th entry from there.

>>> print display_entries(... database[9::10],... [’id’, ’observation_time_start’, ’observation_time_end’, ’instrument’, ’wavemin’, ’wavemax’])id observation_time_start observation_time_end instrument wavemin wavemax-- ---------------------- -------------------- ---------- ------- -------10 2002-02-20 11:06:00 2002-02-20 11:06:43.330000 RHESSI N/A N/A20 N/A N/A N/A N/A N/A30 N/A N/A EIT 25.0 N/A

4.4.4 4. Removing entries

Let’s imagine we want to only have database entries which have some observation time saved. To remove all entrieswhere the value of both observation_time_start and observation_time_end is None, one simply iterates overthe database and uses the Database.remove() method to remove those where the just described predicate is true:

>>> for database_entry in database:... if database_entry.observation_time_start is None and database_entry.observation_time_end is None:... database.remove(database_entry)...>>> len(database)15>>> print display_entries(... database,... [’id’, ’observation_time_start’, ’observation_time_end’, ’instrument’, ’wavemin’, ’wavemax’])id observation_time_start observation_time_end instrument wavemin wavemax-- ---------------------- -------------------- ---------- ------- -------1 2011-03-19 10:54:00.340000 N/A AIA_3 17.1 17.12 2010-10-16 19:12:18 2010-10-16 19:12:22 RHESSI N/A N/A6 2002-06-25 10:00:10.514000 N/A EIT 19.5 19.57 2011-03-19 10:54:00.340000 N/A AIA_3 17.1 17.18 2011-09-22 00:00:00 2011-09-22 00:00:00 BIR N/A N/A10 2002-02-20 11:06:00 2002-02-20 11:06:43.330000 RHESSI N/A N/A21 2012-01-01 00:16:07.836000 N/A SWAP 17.4 17.422 2011-05-08 00:00:00 2011-05-08 00:00:01 AIA 17.1 17.123 2011-05-08 00:00:00 2011-05-08 00:00:01 AIA 21.1 21.124 2011-05-08 00:00:02 2011-05-08 00:00:03 AIA 9.4 9.425 2011-05-08 00:00:03 2011-05-08 00:00:04 AIA 33.5 33.526 2012-08-05 00:00:01 2012-08-05 00:00:02 AIA 9.4 9.427 2012-08-05 00:00:02 2012-08-05 00:00:03 AIA 33.5 33.528 2013-08-05 00:00:01 2013-08-05 00:00:02 AIA 9.4 9.429 2013-08-05 00:00:02 2013-08-05 00:00:03 AIA 33.5 33.5

There are more possible ways to remove entries: You can remove every 2nd entry by iterating over database[::2]and then calling passing every yielded entry to Database.remove(). You can even do advanced operations easilylike for example removing every entry with a certain observation start time, instrument, and FITS header entry pair.



This requires knowledge of the Database.query() method though, which will be covered in section 7, “Querying thedatabase”.

4.4.5 5. Editing entries

5.1 Starring and unstarring entries

The database package supports marking certain entries as “starred”. This concept may be familiar to you from graph-ical applications such as E-Mail clients or photo managers. The method Database.star() marks the passed entry asstarred. Let’s say we are for some reason interested in all values that have a wavelength of 20nm or higher, so we markthose as starred:

>>> for database_entry in database:... if database_entry.wavemin > 20:... database.star(database_entry)...>>> print display_entries(... filter(lambda entry: entry.starred, database),... [’id’, ’observation_time_start’, ’observation_time_end’, ’instrument’, ’wavemin’, ’wavemax’])id observation_time_start observation_time_end instrument wavemin wavemax-- ---------------------- -------------------- ---------- ------- -------23 2011-05-08 00:00:00 2011-05-08 00:00:01 AIA 21.1 21.125 2011-05-08 00:00:03 2011-05-08 00:00:04 AIA 33.5 33.527 2012-08-05 00:00:02 2012-08-05 00:00:03 AIA 33.5 33.529 2013-08-05 00:00:02 2013-08-05 00:00:03 AIA 33.5 33.5

So remove the mark from these entries, the method Database.unstar() works the same way.

5.2 Setting and removing tags

The starring concept is somewhat “binary” in a way that one can only decide whether to mark an entry as starred ornot. To add some more information by assigning keywords to entries, the database package also supports tags. Thetags property of a database object holds all tags that are saved in this database. Let us assign the tag spring to allentries that have been observed in March, April, or May on any day at any year:

>>> for database_entry in database:... if database_entry.observation_time_start.month in [3,4,5]:... database.tag(database_entry, ’spring’)...>>> database.tags[<Tag(name ’spring’)>]>>> spring = database.tags[0]>>> print display_entries(... filter(lambda entry: spring in entry.tags, database),... [’id’, ’observation_time_start’, ’observation_time_end’, ’instrument’, ’wavemin’, ’wavemax’])id observation_time_start observation_time_end instrument wavemin wavemax-- ---------------------- -------------------- ---------- ------- -------1 2011-03-19 10:54:00.340000 N/A AIA_3 17.1 17.17 2011-03-19 10:54:00.340000 N/A AIA_3 17.1 17.122 2011-05-08 00:00:00 2011-05-08 00:00:01 AIA 17.1 17.123 2011-05-08 00:00:00 2011-05-08 00:00:01 AIA 21.1 21.124 2011-05-08 00:00:02 2011-05-08 00:00:03 AIA 9.4 9.425 2011-05-08 00:00:03 2011-05-08 00:00:04 AIA 33.5 33.5



5.3 Changing custom attributes

What sill annoys me a bit is that there are entries in the database where the end of the observation time is None. Letus change them to the same value to the start of the observation time (because we can and because it looks prettier,not because it is accurate). The Database.edit() method receives the database entry to be edited and any number ofkeyword arguments which describe which values to change and how.

>>> for database_entry in database:... if database_entry.observation_time_end is None:... database.edit(database_entry, observation_time_end=database_entry.observation_time_start)...>>> print display_entries(... database,... [’id’, ’observation_time_start’, ’observation_time_end’, ’instrument’, ’wavemin’, ’wavemax’])id observation_time_start observation_time_end instrument wavemin wavemax-- ---------------------- -------------------- ---------- ------- -------1 2011-03-19 10:54:00.340000 2011-03-19 10:54:00.340000 AIA_3 17.1 17.12 2010-10-16 19:12:18 2010-10-16 19:12:22 RHESSI N/A N/A6 2002-06-25 10:00:10.514000 2002-06-25 10:00:10.514000 EIT 19.5 19.57 2011-03-19 10:54:00.340000 2011-03-19 10:54:00.340000 AIA_3 17.1 17.18 2011-09-22 00:00:00 2011-09-22 00:00:00 BIR N/A N/A10 2002-02-20 11:06:00 2002-02-20 11:06:43.330000 RHESSI N/A N/A21 2012-01-01 00:16:07.836000 2012-01-01 00:16:07.836000 SWAP 17.4 17.422 2011-05-08 00:00:00 2011-05-08 00:00:01 AIA 17.1 17.123 2011-05-08 00:00:00 2011-05-08 00:00:01 AIA 21.1 21.124 2011-05-08 00:00:02 2011-05-08 00:00:03 AIA 9.4 9.425 2011-05-08 00:00:03 2011-05-08 00:00:04 AIA 33.5 33.526 2012-08-05 00:00:01 2012-08-05 00:00:02 AIA 9.4 9.427 2012-08-05 00:00:02 2012-08-05 00:00:03 AIA 33.5 33.528 2013-08-05 00:00:01 2013-08-05 00:00:02 AIA 9.4 9.429 2013-08-05 00:00:02 2013-08-05 00:00:03 AIA 33.5 33.5

You may ask yourself now “Why can’t I simply use database_entry.observation_time_end =database_entry.observation_time_start”? Well, the answer is: you can, but it has one major disadvan-tage: you cannot undo this operation if you don’t use the methods of the database objects such as Database.edit.See the section 6, “Undoing and redoing operations”, to see how undoing and redoing works.

4.4.6 6. Undoing and redoing operations

A very handy feature of the database package is that every operation can be reverted that changes the database insome way. The Database class has the methods Database.undo() and Database.redo() to undo and redo the last ncommands, respectively.

Note: The undo and redo history are only saved in-memory! That means in particular, that if you work on a Databaseobject in an interactive Python session and quit this session, the undo and redo history are lost.

In the following snippet, the operations from the sections 5.3, 5.2, and 5.1 are undone. Note the following changes:there are no longer any tags anymore saved in the database, there is no entry which is starred and there are again entrieswith no end of observation time.

>>> database.undo(4) # undo the edits from 5.3 (4 records have been affected)>>> database.undo(6) # undo tagging some entries with the tag ’spring’>>> database.undo(4) # undo starring entries>>> print display_entries(... database,... [’id’, ’observation_time_start’, ’observation_time_end’, ’instrument’, ’wavemin’, ’wavemax’, ’tags’, ’starred’])


http://docs.python.org/library/constants.html#None


id observation_time_start observation_time_end instrument wavemin wavemax tags starred-- ---------------------- -------------------- ---------- ------- ------- ---- -------1 2011-03-19 10:54:00.340000 N/A AIA_3 17.1 17.1 N/A No2 2010-10-16 19:12:18 2010-10-16 19:12:22 RHESSI N/A N/A N/A No6 2002-06-25 10:00:10.514000 N/A EIT 19.5 19.5 N/A No7 2011-03-19 10:54:00.340000 N/A AIA_3 17.1 17.1 N/A No8 2011-09-22 00:00:00 2011-09-22 00:00:00 BIR N/A N/A N/A No10 2002-02-20 11:06:00 2002-02-20 11:06:43.330000 RHESSI N/A N/A N/A No21 2012-01-01 00:16:07.836000 N/A SWAP 17.4 17.4 N/A No22 2011-05-08 00:00:00 2011-05-08 00:00:01 AIA 17.1 17.1 N/A No23 2011-05-08 00:00:00 2011-05-08 00:00:01 AIA 21.1 21.1 N/A No24 2011-05-08 00:00:02 2011-05-08 00:00:03 AIA 9.4 9.4 N/A No25 2011-05-08 00:00:03 2011-05-08 00:00:04 AIA 33.5 33.5 N/A No26 2012-08-05 00:00:01 2012-08-05 00:00:02 AIA 9.4 9.4 N/A No27 2012-08-05 00:00:02 2012-08-05 00:00:03 AIA 33.5 33.5 N/A No28 2013-08-05 00:00:01 2013-08-05 00:00:02 AIA 9.4 9.4 N/A No29 2013-08-05 00:00:02 2013-08-05 00:00:03 AIA 33.5 33.5 N/A No

The redo method reverts the last n operations that have been undone. If not that many operations can be redone (i.e.any number greater than 14 in this example), an exception is raised. You can see that the redo call reverts the originalstate: the tags appeared again and all entries with a wavelength >20nm are starred again. Also, there are no entrieswith no stored end of observation time anymore.

>>> database.redo(14) # redo all undone operations>>> print display_entries(... database,... [’id’, ’observation_time_start’, ’observation_time_end’, ’instrument’, ’wavemin’, ’wavemax’, ’tags’, ’starred’])id observation_time_start observation_time_end instrument wavemin wavemax tags starred-- ---------------------- -------------------- ---------- ------- ------- ---- -------1 2011-03-19 10:54:00.340000 2011-03-19 10:54:00.340000 AIA_3 17.1 17.1 spring No2 2010-10-16 19:12:18 2010-10-16 19:12:22 RHESSI N/A N/A N/A No6 2002-06-25 10:00:10.514000 2002-06-25 10:00:10.514000 EIT 19.5 19.5 N/A No7 2011-03-19 10:54:00.340000 2011-03-19 10:54:00.340000 AIA_3 17.1 17.1 spring No8 2011-09-22 00:00:00 2011-09-22 00:00:00 BIR N/A N/A N/A No10 2002-02-20 11:06:00 2002-02-20 11:06:43.330000 RHESSI N/A N/A N/A No21 2012-01-01 00:16:07.836000 2012-01-01 00:16:07.836000 SWAP 17.4 17.4 N/A No22 2011-05-08 00:00:00 2011-05-08 00:00:01 AIA 17.1 17.1 spring No23 2011-05-08 00:00:00 2011-05-08 00:00:01 AIA 21.1 21.1 spring Yes24 2011-05-08 00:00:02 2011-05-08 00:00:03 AIA 9.4 9.4 spring No25 2011-05-08 00:00:03 2011-05-08 00:00:04 AIA 33.5 33.5 spring Yes26 2012-08-05 00:00:01 2012-08-05 00:00:02 AIA 9.4 9.4 N/A No27 2012-08-05 00:00:02 2012-08-05 00:00:03 AIA 33.5 33.5 N/A Yes28 2013-08-05 00:00:01 2013-08-05 00:00:02 AIA 9.4 9.4 N/A No29 2013-08-05 00:00:02 2013-08-05 00:00:03 AIA 33.5 33.5 N/A Yes

4.4.7 7. Querying the database

The API for querying databases is similar to querying the VSO using the methodsunpy.net.vso.VSOClient.query(). The Database.query() method accepts any number of ORed queryattributes (using |) and combines them using AND. It returns a list of matched database entries. The special thingabout querying databases is that all attributes support the unary operator ~ to negate specific attributes. Example: thequery ~Instrument(’EIT’) returns all entries that have not been observed with the EIT.



7.1 Using VSO attributes

Using the attributes from sunpy.net.vso.attrs is quite intuitive: the simple attributes and the Time attribute workexactly as you expect it. Note though that the near parameter of sunpy.net.vso.attrs.Time is ignored! The reasonfor this is that its behaviour is not documented and that it is different depending on the server which is requested. Thefollowing query returns the data that was added in section 2.3.2, “Downloading”:

>>> print display_entries(... database.query(vso.attrs.Time(’2012-08-05’, ’2012-08-05 00:00:05’), vso.attrs.Instrument(’AIA’)),... [’id’, ’observation_time_start’, ’observation_time_end’, ’instrument’, ’wavemin’, ’wavemax’])id observation_time_start observation_time_end instrument wavemin wavemax-- ---------------------- -------------------- ---------- ------- -------26 2012-08-05 00:00:01 2012-08-05 00:00:02 AIA 9.4 9.427 2012-08-05 00:00:02 2012-08-05 00:00:03 AIA 33.5 33.5

When using the sunpy.net.vso.attrs.Wave attribute, you have to know that the third parameter (unit) is ‘angstrom’per default. So if it was not passed in the following example, the database would have been queried for all entries witha wavelength from 1 nm to 2 nm (10 Ångstrom are 1 nm) and therefore, an empty list would have been returned.

>>> print display_entries(... database.query(vso.attrs.Wave(10, 20, ’nm’)),... [’id’, ’observation_time_start’, ’observation_time_end’, ’instrument’, ’wavemin’, ’wavemax’])id observation_time_start observation_time_end instrument wavemin wavemax-- ---------------------- -------------------- ---------- ------- -------1 2011-03-19 10:54:00.340000 2011-03-19 10:54:00.340000 AIA_3 17.1 17.1 spring No6 2002-06-25 10:00:10.514000 2002-06-25 10:00:10.514000 EIT 19.5 19.5 N/A No7 2011-03-19 10:54:00.340000 2011-03-19 10:54:00.340000 AIA_3 17.1 17.1 spring No21 2012-01-01 00:16:07.836000 2012-01-01 00:16:07.836000 SWAP 17.4 17.4 N/A No22 2011-05-08 00:00:00 2011-05-08 00:00:01 AIA 17.1 17.1 spring No

7.2 Database-specific attributes

There are 5 additional query attributes supported by the database package. They can be imported from the submodulesunpy.database.attrs and are in particular:

• Starred

• Tag

• Path

• DownloadTime

• FitsHeaderEntry

The following query searches for all entries that have the tag ‘spring’ or (inclusive or!) are starred and have not theFITS header key ‘WAVEUNIT’ with the value ‘Angstrom’:

>>> print display_entries(... database.query(dbattrs.Tag(’spring’) | dbattrs.Starred(), ~dbattrs.FitsHeaderEntry(’WAVEUNIT’, ’Angstrom’)),... [’id’, ’observation_time_start’, ’observation_time_end’, ’instrument’, ’wavemin’, ’wavemax’, ’tags’, ’starred’])id observation_time_start observation_time_end instrument wavemin wavemax tags starred-- ---------------------- -------------------- ---------- ------- ------- ---- -------7 2011-03-19 10:54:00.340000 2011-03-19 10:54:00.340000 AIA_3 17.1 17.1 spring No1 2011-03-19 10:54:00.340000 2011-03-19 10:54:00.340000 AIA_3 17.1 17.1 spring No22 2011-05-08 00:00:00 2011-05-08 00:00:01 AIA 17.1 17.1 spring No23 2011-05-08 00:00:00 2011-05-08 00:00:01 AIA 21.1 21.1 spring Yes24 2011-05-08 00:00:02 2011-05-08 00:00:03 AIA 9.4 9.4 spring No25 2011-05-08 00:00:03 2011-05-08 00:00:04 AIA 33.5 33.5 spring Yes



27 2012-08-05 00:00:02 2012-08-05 00:00:03 AIA 33.5 33.5 N/A Yes29 2013-08-05 00:00:02 2013-08-05 00:00:03 AIA 33.5 33.5 N/A Yes

4.4.8 8. Caching

All entries that are saved in the database are also saved in a cache in-memory. The type of the cache is determinedat the initialization of the database object and cannot be changed after that. The default type is caching.LRUCache(least-recently used) and the other one which is supported is caching.LFUCache (least-frequently used). Per defaultthe cache size is float(’inf’), i.e. infinite. To set the cache size after the database object has been initialized, use themethod Database.set_cache_size(). If the new size is smaller than the current number of database entries, entriesare removed according to the cache type until the number of entries is equal to the given cache size. The followingcall to Database.set_cache_size() sets the cache size to 10 and therefore removes the 5 entries that been used leastrecently.

>>> database.set_cache_size(10)>>> print display_entries(... database,... [’id’, ’observation_time_start’, ’observation_time_end’, ’instrument’, ’wavemin’, ’wavemax’])id observation_time_start observation_time_end instrument wavemin wavemax-- ---------------------- -------------------- ---------- ------- -------1 2011-03-19 10:54:00.340000 2011-03-19 10:54:00.340000 AIA_3 17.1 17.16 2002-06-25 10:00:10.514000 2002-06-25 10:00:10.514000 EIT 19.5 19.57 2011-03-19 10:54:00.340000 2011-03-19 10:54:00.340000 AIA_3 17.1 17.121 2012-01-01 00:16:07.836000 2012-01-01 00:16:07.836000 SWAP 17.4 17.423 2011-05-08 00:00:00 2011-05-08 00:00:01 AIA 21.1 21.125 2011-05-08 00:00:03 2011-05-08 00:00:04 AIA 33.5 33.526 2012-08-05 00:00:01 2012-08-05 00:00:02 AIA 9.4 9.427 2012-08-05 00:00:02 2012-08-05 00:00:03 AIA 33.5 33.528 2013-08-05 00:00:01 2013-08-05 00:00:02 AIA 9.4 9.429 2013-08-05 00:00:02 2013-08-05 00:00:03 AIA 33.5 33.5

4.5 Opening Files with SunPy

SunPy has wrapped several libraries in order to make input and output as painless as possible. So let me take you on ajourney through the binary forests using the chainsaw of SunPy.

4.5.1 1. Wrapped Libraries

Currently three IO libraries are wrapped for easy use with SunPy.

1. AstroPy’s Fits

2. Glymur’s JPEG 2000 C library wrapper

3. ANA C library

Luckily for everyone, SunPy has general purpose high-level IO functions.:

import sunpy.io as ioio.file_tools.read_file(filename)io.file_tools.read_file_header(filename)io.file_tools.write_file(filename)



These are designed to take a filename and breakdown the name in order to automatically calculate the call requiredto either read or write this file. SunPy has a “database” of file extensions which is compared against when using thehigh-level IO functions. When reading or writing a data file, the function will return a list of (data, header) pairsdepending on how HDUs exist in the file. It important to remember this. If for some reason these calls fail, which ispossible to do unknown file extensions or unsupported file types, create an issue and someone will get back to you.

Further, you can force the filetype from this interface, like so:

io.fileTools.read_file(filename,filetype= ’filetype of your choice’)

This will work for the three high-level IO functions.

Sometimes, you require fine control over your data files. Come on and follow me deeper.

1.1 Fits files

SunPy’s Fits reading ability comes directly from AstroPy, as they swallowed PyFits whole a while back now.

Currently four functions exist under the io.fits namespace.:

import sunpy.io.fits as fitsfits.read(filename)fits.write(filename, data, header)fits.get_header(filename)fits.extract_waveunit(header)

So, the functions here are basic. You can read/write to/from fit files but also get only the header as well. The finalfunction is used to figure out the wave unit from a fits header and so you should pass a header object into this function.

Full documentation is located here.

1.2 JPEG 2000 files

SunPy’s JPEG reading ability comes from wrappying Glymur which itself wraps a OpenJPEG 2000 C library. Lots ofwrapping here. Christmas come early.

Currently four functions exist under the io.jp2 namespace.:

import sunpy.jp2 as jp2jp2.read_file(filename)jp2.read_file_header(filename)jp2.write_file(filename)


1.3 fz files

SunPy’s ANA (fz) reading ability comes from wrapping a C library designed to read and write ANA compressed files.It is common to find fits files compressed this way. The fits wrapper already takes care of these files.

Currently four functions exist under the io.ana namespace.:

import sunpy.io.ana as anaana.read_file(filename)ana.read_file_header(filename)ana.write_file(filename)


4.5. Opening Files with SunPy 51


1.4 Other files

For these, you are on your own. However, Python has many IO libraries and there will be the ability to read them in.You can join the IRC channel for support on freenode (#sunpy) or the mailing list (link) and the github issues section(link).


CHAPTER 5

Data Types in SunPy

5.1 Maps

Maps in SunPy are dimensionally-aware data arrays. In other words, they are 2-dimensional data associated with acoordinate system. In this guide, we will cover some of the basic functionality of maps. Once you’ve read throughthis guide check out the api reference for a more thorough look at SunPy maps.

5.1.1 Data Support

The map object currently supports the following data sources

• SDO/AIA, SDO/HMI

• STEREO/EUVI, STEREO/COR

• Hinode/XRT

• SOHO/EIT, SOHO/LASCO, SOHO/MDI

• PROBA2/SWAP

• Yohkoh/SXT

1. Creating maps

SunPy contains a number of example fits files. To make things easy, SunPy includes several example files which areused throughout the docs. These files have names like sunpy.AIA_171_IMAGE and sunpy.RHESSI_IMAGE. To createthe sample AIA map type the following into your interactive Python shell:

import sunpymy_map = sunpy.Map(sunpy.AIA_171_IMAGE)

The variable my_map is a SunPy Map object. To create a SunPy Map object from a local fits file try something likethe following

my_map = sunpy.Map(’/mydirectory/mymap.fits’)

53


SunPy automatically detects the type of file (e.g. fits), what instrument it is associated with (e.g. AIA, EIT, LASCO)and will automatically look in the appropriate places for the fits keywords it needs to interpret the coordinate system.If the type of fits file is not recognized then SunPy will try some default fits keywords and return a GenericMap butresults may vary. SunPy can also create maps from the jpg2000 files from helioviewer.org.

2. Creating Custom Maps

It is also possible to create maps using custom data from a simulation for example. To do this you need to provideMap() with both the data array as well as some basic meta information. If no header is given then some default valuesas assumed. Here is a simple example:

import numpy as npdata = np.arange(0,100).reshape(10,10)header = {’cdelt1’: 10, ’cdelt2’: 10, ’telescop’:’sunpy’}my_map = sunpy.Map(data, header)

The format of the header follows the fits standard.

2. Inspecting maps

A map contains a number of data-associated attributes. To get a quick look at your map simply type:

my_map = sunpy.Map(sunpy.AIA_171_IMAGE)my_map

This will show a representation of the data as well as some of its associated attributes. A number of other attributesare also available, for example the date, exposure time, map center, xrange, yrange other:

map_date = my_map.datemap_exptime = my_map.exposure_timemap_center = my_map.centermap_xrange = my_map.xrangemap_yrange = my_map.yrange

To get a list of all of the attributes check the documentation by typing:

help(my_map)

The meta data for the map is accessed by

header = my_map.meta

This references the meta data dictionary with the header information as read from the source file.

3. Getting at the data

The data in a SunPy Map object is accessible through the data attribute. Currently, the data is implemented as a NumPyndarray, so for example, to get the 0th element in the array

my_map.data[0,0]my_map.data[0][0]

One important fact to remember which is intially confusing is that the first index is for the y direction while the secondindex is for the x direction! For more information about indexing please refer to the Numpy documentation. Commonndarray attributes, such as shape and dtype, are accessible through the SunPy Map object

54 Chapter 5. Data Types in SunPy


my_map.shapemy_map.dtype

If you’d like to use the data in a SunPy Map object elsewhere, you can use

var = my_map.data# orvar = my_map.data.copy()

Basic statistical functions on the data array are also passed through to Map objects:

my_map.min()my_map.max()my_map.mean()

4. Creating a plot of your map

The SunPy map object has its own built-in plot methods so that it is easy to quickly view your map on the screen. Tocreate a plot just type:

my_map.peek()

This will open a matplotlib plot right on your screen. In addition, to enable users to modify the plot it is possible tograb the matplotlib figure object by using the plot() command instead of the show() command. This makes it possibleto use the SunPy plot as the foundation for a more complicated figure.

5. Overlaying Maps

The Map() method described above can also handle a list of maps. If a list in inputs is supplied, Map() will return a listof maps as the output. However, if the ‘composite’ keyword is set to True, then a CompositeMap object is returned.This is useful if the maps are of a different type (e.g. different instruments). For example, to create a simple compositemap:

my_maps = sunpy.Map(sunpy.EIT_195_IMAGE, sunpy.RHESSI_IMAGE, composite=True)

A CompositeMap is different from a regular SunPy Map objectand therefore different associated methods. To listwhich maps are part of your composite map use:

my_maps.list_maps()

Similar to all SunPy data objects, the composite map also has an associated show() method and a number of associatedmethods to customize your plot. For example, the following code turns adds a new map (which must be instantiatedfirst), sets its transparency to 25%, turns on contours from 50% to 90% for the second map, and then plots the result:

my_maps.add_map(sunpy.Map(sunpy.AIA_171_IMAGE))my_maps.set_alpha(2,0.5)my_maps.set_levels(1,[50,60,70,80,90], percent = True)my_maps.peek()

This is not a particularly pretty plot but it shows what SunPy can do!

5. Working with your map

Part of the philosophy of the map object is to provide most of the basic functionality that a scientist would wanttherefore a map also contains a number of map-specific methods such as resizing a map or grabbing a subview. To geta list of the methods available for a map type:

5.1. Maps 55


help(my_map)

and check out the methods section!

5.2 Lightcurves

Time series data are a fundamental part of many data analysis projects as much in heliophysics as other areas. SunPytherefore provides a lightcurve object to deal with this data type. Thankfully most of the heavy lifting in this areahas already been done for us. The lightcurve object makes use of the pandas python module. Pandas is a high-quality and optimized module which is in use in a wide variety of academic and commercial fields, including Finance,Neuroscience, Economics, Statistics, Advertising, and Web Analytics. The lightcurve object is essentially a wrapperaround a pandas dataframe object which holds some of the meta-data from the original data source. In this tutorialwe provide a quick introduction to the lightcurve object and pandas. We highly recommend any user of the lightcurveobject take a look at the great pandas documentation. for more information.

5.2.1 Data Support

The lightcurve object currently supports the following data sources

• SDO/EVE

• GOES XRS

• PROBA2/LYRA

5.2.2 1. Creating a Lightcurve

A LightCurve object consists of two parts - times, and measurements which were taken at those times.

A LightCurve object must be supplied with some data when it is created. The data can either be in your current Pythonsession, in a local file, or in a remote file. Let’s create some fake data and pass it into a LightCurve object.

>>> from sunpy.lightcurve import LightCurve>>> light_curve = LightCurve.create({"param1": range(24 * 60)})

The first line imports the lightcurve object. Let’s look at the argument in LightCurve.create. The argument is adictionary that contains a single entry with key “param1” with a value of a list of 1440 entries (from 0 to 1439) - theseare our ‘fake data’ measurements. Since no other times are provided, a default set of times are provided. You canprovide your own times very simply using the ‘index’ keyword, as is shown below.

>>> import datetime>>> base = datetime.datetime.today()>>> dates = [base - datetime.timedelta(minutes=x) for x in range(0, 24 * 60)]>>> light_curve = LightCurve.create({"param1": range(24 * 60)}, index=dates)

This gives the measurements “param1” a set of times, in this case, 1440 minutes beginning at the current local time.

The LightCurve object contains two basic attributes, ‘data’ and ‘header’. The ‘data’ attribute is either a pandas Time-Series object or a pandas DataFrame (a generalization of the TimeSeries object). These data objects have very powerfulmethods for handling data based on times.


http://pandas.pydata.org/

http://pandas.pydata.org/pandas-docs/stable/


5.3 Spectra

Warning: This module is under development! Use at your own risk.

5.3.1 Spectrograms

SunPy currently supports reading dynamic spectra from e-Callisto instruments. The main class that is used for thisis CallistoSpectrogram. SunPy also comes with an example image that shows a radio burst observed at RosseObservatory (aka. BIR; Birr Castle, Co. Offaly, Ireland) that can be found in sunpy.CALLISTO_IMAGE:

from matplotlib import pyplot as pltimport sunpyfrom sunpy.spectra.sources.callisto import CallistoSpectrogramimage = CallistoSpectrogram.read(sunpy.CALLISTO_IMAGE)

You can now view the image by using the peek() method.

image.peek()

We now notice that there seems to be something interesting that has been cut off at the corner of the image, so we usethe extend method to request more data from the server. It optionally takes the amount of minutes we want to requestfrom the server (negative values mean we want to add data that was registered before our existing local data), if noneare given it defaults to 15 minutes (the size of one e-Callisto file).:

5.3. Spectra 57

http://www.e-callisto.org/

http://rosseobservatory.ie/

http://rosseobservatory.ie/


more = image.extend()more.peek()

We will, for the purposes of this demonstration, continue working with the original image, though.

You can then perform automatic constant background subtraction by using the subtract_bg() method. The resultingimage will be clipped at 0 using the min parameter of peek in order to avoid negative values.:

nobg = image.subtract_bg()nobg.peek(min_=0)

If you want to see the background determined by the automatic subtraction, you can use the auto_const_bg() methodand visualize the resulting data using pyplot.plot().:

plt.figure()bg = image.auto_const_bg()plt.plot(image.freq_axis, bg)


http://docs.python.org/library/functions.html#min


plt.xlabel("Frequency [MHz]")plt.ylabel("Intensity")plt.show() # This might not be necessary if you are using pylab.

Now let us say we want to isolate the interesting bit (which starts around 10:38) from the boring background; there isa method called in_interval() that allows us to take the part of an image that is within a specified interval. Leavingout the second argument it defaults to the end time of the file.:

interesting = nobg.in_interval("10:38")interesting.peek(min_=0)

To get rid of the noise, we could also clip low intensities.:

5.3. Spectra 59


interesting.peek(min_=20)

If we want more context, we can also join together different images into a large one in time (note that this does morethan just concatenating the array and the axes – it also considers possible overlap or gaps).:

c1 = CallistoSpectrogram.read("BIR_20110922_101500_01.fit")c2 = CallistoSpectrogram.read("BIR_20110922_103000_01.fit")d = CallistoSpectrogram.join_many([c1, c2])d.peek()

We could also get the from_range method to get data between those two points directly from the archive and joinedtogether (though that will fetch all frequencies of BIR).:

d = CallistoSpectrogram.from_range("BIR", "2011-09-22T10:15:00", "2011-09-22T10:45:00")d.peek()



The peek() method returns a special kind of figure that offers convenience features needed for analyzing spectrograms,a SpectroFigure. By using its time_freq(), we can select points on the figure and frequency-time information ofthem will be returned as a TimeFreq. Time is stored as offsets in seconds from TimeFreq.start. Note that if you useplot() instead of plot(), you have to create a SpectroFigure by using SpectroFigure:

fig = image.peek()time_freq = fig.time_freq()# Select points.time_freq.time#array([ 0. , 54.5 , 104. , 163.25])time_freq.freq#array([ 68.76923077, 59.29888786, 48.50092678, 36.46385542])

We can then quickly visualize this using the peek() method:

time_freq.peek(marker=’o’, linestyle=’--’)

5.3. Spectra 61

CHAPTER 6

Plotting in SunPy

SunPy makes use of matplotlib for all of its plotting needs as such tries to follow the matplotlib plotting philosophy. Itis therefore useful to go over how matplotlib works as background.

6.1 1. Matplotlib Tutorial

The tutorial provided here is a summary of one that can be found in the matplotlib usage documentation.

Matplotlib provides two main pathways for plotting. One is meant for interactive use (e.g. command-line) and the otherfor non-interactive use (e.g. modules). It is important to recognize though that the interactive-use pathway (referredto as pyplot) just provides shortcuts doing many of the more advanced non-interactive functions in the background.It is therefore possible to switch between the two as necessary and convenient and it is possible to use pyplot in non-interactive way. In this manner pyplot is just a shortcut making it quicker to set up plot axes and figures. In order to getaccess to the full interactive capabilities of pyplot it is necessary to turn this feature on. This will be discussed below.Pylab is another matplotlib usage scenario but it is essentially just pyplot with the interactive capabilities turned onand numpy and matplotlib imported into the main namespace.

6.2 2. Pyplot

Here is an example of pyplot usage

import matplotlib.pyplot as plt

plt.plot(range(10), range(10))plt.title("A simple Plot")plt.show()

The show() command open a plot on the screen and blocks execution (meaning you can’t do anything with the promptor your script freezes) until the plot window is closed. For reasons that are not very clear, the creators of matplotlibdesigned the show() command so that it would only work once. If you were to call show() on the plt object againafter the above code is executed nothing happens. Apparently, this confusing behavior is something that the matplotlibdevs get complaints about often and so this may change in the future (or may already have changed depending on yourchoice of backend). A discussion about this can be found here. Don’t be confused by another command called draw().This is only used while in interactive mode.

To turn on interactivity for pyplot use the command

63

http://matplotlib.org/

http://matplotlib.org/faq/usage_faq.html

http://stackoverflow.com/questions/5524858/matplotlib-show-doesnt-work-twice


plt.ion()

In interactive mode, the plot will appear at the first plot() command and most commands will update the plot as youcall them. Here is an example

plt.plot(range(10), range(10))plt.title("Simple Plot")

In this example, you’ll see that the title appears right on the plot when you call it. Note that in this case the show com-mand is useless as the plot shows up right when you create it. Also note that some commands will not automaticallyupdate the plot and you have to use the draw() command. The following command

plt.ioff()

turns off interactivity.

6.3 3. Advanced Pyplot

If you need more fine-grained control over plots the recommended path is to use pyplot and access the figures and axesobjects. Here is an example

import matplotlib.pyplot as pltimport numpy as np

x = np.arange(0, 10, 0.2)y = np.sin(x)

fig = plt.figure()ax = fig.add_subplot(111)ax.plot(x, y)

plt.show()

Figure is the top-level container for all plot elements and axes is the top-level container for a particular plot. So theabove example, creates a figure then creates an axes and then uses the pyplot plot() method to populate the plot inax. You generally don’t need to mess with the figure object but with this method you now have your hands on the axobject so you can do this like change the labels on the x and y axes or add a legend, etc. do whatever you want to it. Inthe previous section, pyplot took care of creating these objects for you so you don’t have to worry about creating themyourself.

6.4 4. SunPy Plotting Standards

To be consistent with matplotlib, SunPy has developed a standard plotting policy which supports both simple andadvanced matplotlib usage.

6.5 5. peek()

For quick and easy access to a plot all sunpy base objects (e.g. map, spectra, lightcurve) define their own peek()command. For example you can do the following

64 Chapter 6. Plotting in SunPy


import sunpy

smap = sunpy.Map(sunpy.EIT_195_IMAGE)smap.peek(draw_limb=True)

This creates a plot window with all axes defined, a plot title, and the image of the map data all defined by the contentsof the map. As this is command makes use of show(), in non-interactive mode the plot window blocks and must beclosed before doing anything else. This is meant as a quick way to visualize the contents of a sunpy object you’vecreated.

6.6 6. plot()

For more advanced plotting the base sunpy objects also provide a plot() command. This command is similar to thepyplot plot() command in that it will create a figure and axes object for you if you haven’t already. It returns a figureobject and does not create a plot window. With the figure object in your hands you can reach in and grab the axes andtherefore manipulate the plot as you see fit. Here is an example of this at work

import sunpyimport matplotlib.pyplot as plt

smap = sunpy.Map(sunpy.EIT_195_IMAGE)smap.plot()smap.draw_limb()

plt.show()

This output of this example is equivalent to one in the previous section. The map.plot() command is equivalent toax.plot(x,y) command which we introduced in section 3. Similar to that command it will create a figure for you if youhaven’t created on yourself. For advanced plotting you’ll want to create it yourself.

fig = plt.figure()ax = plt.subplot(1,1,1)

smap.plot()plt.colorbar()ax.plot([-1000,1000], [0,0], color="white")

plt.show()

The above a plot of line across the map. Using the fig.gca() command to get access to the axes object most anythingcan be done to the plot and the plot can be displayed as usual using the show() command. Here is another example

from matplotlib import patchesfig = plt.figure()ax = plt.subplot(1,1,1)

smap.plot()rect = patches.Rectangle([-350, -650], 500, 500, color = ’white’, fill=False)ax.add_artist(rect)

plt.show()

Finally, here is a more complex example, starting from the beginning:

from matplotlib import patchesimport sunpy

6.6. 6. plot() 65


import matplotlib.pyplot as pltsmap = sunpy.Map(sunpy.AIA_171_IMAGE)submap = smap.submap([-100-250, -100+250], [-400-250, -400+250])rect = patches.Rectangle([-100-250, -400-250], 500, 500, color = ’white’, fill=False)

fig = plt.figure()ax1 = fig.add_subplot(2,1,1)smap.plot()ax1.add_artist(rect)

ax2 = fig.add_subplot(2,1,2)submap.plot()submap.draw_grid(grid_spacing=10)ax2.set_title(’submap’)fig.subplots_adjust(hspace=0.4)

plt.show()

The above example creates two side by side plots one with the overall view of the Sun with a small area marked witha white box. That smaller view is then shown in the plot below it. The spacing between the two plots is controlled byfig.subplots_adjust().

6.7 7. Specifying a Colormap

There are a number of color maps defined in SunPy which are used for data from particular missions (e.g. SDO/AIA).A simple example on how to use the color maps provided by SunPy:

from sunpy.cm import cm

# cmlist is a dictionary with all of the color tables# to list all of the keys of the dictionarycm.cmlist.keys()

# to grab a particular colortable thencmap = cm.cmlist.get(’sdoaia94’)

# you can also get a visual representation of all of the color tablescm.show_colormaps()



These can be used with the standard commands to change the colormap:

import sunpyfrom sunpy.cm import cm

cmap = cm.cmlist.get(’sdoaia94’)smap = sunpy.Map(sunpy.AIA_171_IMAGE)

fig = plt.figure()ax = plt.subplot(1,1,1)smap.plot(cmap=cmap)plt.show()

6.7. 7. Specifying a Colormap 67

CHAPTER 7

Time in SunPy

Working with times and time ranges is a standard task in solar data analysis as such SunPy strives to provide convenientand easy methods to do the simple stuff. Python already provides an object for a time or date through the datetimeobject. SunPy builds upon its functionality.

7.1 1. Parsing Times

Solar data is associated with a number of different time formats. SunPy provides a simple parsing function which candeal with most every format that a user may encounter. Called parse_time(), this function takes a string as input andreturns a datetime object. Here are few examples of formats which parse_time() accepts:

from sunpy.time import *parse_time(’2007-05-04T21:08:12’)parse_time(’2007/05/04T21:08:12’)parse_time(’20070504T210812’)parse_time(’2007-May-04 21:08:12’)parse_time(’20070504_210812’)

Each of the above returns the same datetime object. All SunPy functions which require time as an input sanitize theinput using parse_time. You can also pass it a datetime object directly and it will simply hand it right back to you. Forusers of IDL, this function is meant to the be the equivalent to SSW’s anytim().

7.2 2. Time Ranges

A very standard task in data analysis is to have to deal with pairs of times or time ranges. This occurs very often withplotting or when searching for data. To deal with time ranges SunPy provides the TimeRange object. A time rangeobject can be created very easily by providing it with two time strings, a start time and an end time:

time_range = TimeRange(’2010/03/04 00:10’, ’2010/03/04 00:20’)

You can also pass the start and end times as a tuple:

time_range = TimeRange((’2010/03/04 00:10’, ’2010/03/04 00:20’))

69

http://docs.python.org/2/library/datetime.html

http://docs.python.org/2/library/datetime.html


This object makes use of parse_time() so it can accept a wide variety of time formats. A time range object can also becreated by providing a start time and a duration. The duration must be provided as a timedelta object or as a numberof seconds so for example:

time_range = TimeRange(’2010/03/04 00:10’, 400)

or:

from datetime import timedeltatime_range = TimeRange(’2010/03/04 00:10’, timedelta(0,400))

The time range objects provides a number of useful functions. For example, you can easily get the time at the centerof your interval or the length of your interval in minutes or days or seconds:

time_range.center()time_range.minutes()time_range.days()time_range.seconds()

It also makes it easy to create new time ranges. The functions next() and previous() do an inplace update to the objectby either adding or subtracting the same time interval . This could be useful if you need to step through a number oftime ranges. For example, if you needed time ranges that spanned 30 minutes over a period of 4 hours you could do:

for a in range(8): print(time_range.next())

Check out the code reference for the time range object for more information.

70 Chapter 7. Time in SunPy

http://docs.python.org/2/library/datetime.html#datetime.timedelta

CHAPTER 8

Region of Interest

An region of interest (ROI) is an object that contains some basic information about a particular time range in theform of string descriptors. For example, an ROI might denote an interval of troublesome instrument data, such as anencounter with the South Antarctic Anomaly (SAA).

8.1 1. Creating an ROI

You can create an ROI object with the following:

from sunpy.roi import *result = roi(times=[’2011-02-15 04:34:09’,’2011-02-15 04:48:21’],description=’UV occult.’,source=’LYRA LYTAF’)

This creates an roi called result for the specific time range. Querying the newly created ROI gives the following:

In [4]: resultOut[4]:SunPy Region-of-interest (ROI) object-------------------------------------Source: LYRA LYTAFStart time: 2011-02-15T04:34:09End time: 2011-02-15T04:48:21Event description: UV occult.

Check out the code reference for the time range object for more information.

71


72 Chapter 8. Region of Interest

CHAPTER 9

Customizing sunpy

9.1 The sunpyrc file

Sunpy uses sunpyrc configuration files to customize certain properties. You can control a number of key features ofSunPy such as where your data will download to. SunPy looks for sunpyrc in two locations, in the following order:

1. .sunpy/sunpyrc, for the user’s default customizations.

2. INSTALL/sunpy/data/sunpyrc, where INSTALL is something like /usr/lib/python2.5/site-packages onLinux, and maybe C:\Python25\Lib\site-packages on Windows. Every time you install SunPy, this file willbe overwritten, so if you want your customizations to be saved, please move this file to your .sunpyrc directory.

To display where the currently active sunpyrc file was loaded from, one can do the following:

>>> import sunpy>>> sunpy.print_config()

The default sunpyrc file lives in sunpy/data/. We recommend that you make a copy of this file and put it where Sunpywill find it. See below for the example config file.

9.2 Dynamic settings

You can also dynamically change the default settings in a python script or interactively from the python shell. All ofthe settings are stored in a Python ConfigParser instance called sunpy.config, which is global to the sunpy package.Settings can be modified directly, for example:

import sunpysunpy.config.set(’downloads’, ’download_dir’, ’/home/user/Downloads’)

9.2.1 A sample sunpyrc file

;; SunPy Configuration;; This is a sample sunpy configuration file - you can find a copy; of it on your system in site-packages/sunpy/data/sunpyrc. If you edit it

73


; there, please note that it will be overridden in your next install.; If you want to keep a permanent local copy that will not be; over-written, place it in HOME/.sunpy/sunpyrc (unix/linux; like systems) and C:\Documents and Settings\yourname\.sunpy; (win32 systems).;; Note that any relative filepaths specified in the SunPy configuration file; will be relative to SunPy’s working directory.;

;;;;;;;;;;;;;;;;;;;; General Options ;;;;;;;;;;;;;;;;;;;;[general]

; The SunPy working directory is the parent directory where all generated; and download files will be stored.; Default Value: <user’s home directory>/sunpy; working_dir = /home/$USER/sunpy

;;;;;;;;;;;;;; Downloads ;;;;;;;;;;;;;;[downloads]

; Location to save download data to. Path should be specified relative to the; SunPy working directory.; Default value: data/;download_dir = /tmpdownload_dir = data

74 Chapter 9. Customizing sunpy

CHAPTER 10

Troubleshooting

Contents

• Troubleshooting– Crotate Warning– Obtaining sunpy version– System Info– sunpy install location– .sunpy directory location– Report a problem

10.1 Crotate Warning

The SunPy map class has a custom rotate functionality, similar to IDL’s ROT function. This uses a Python C-APIextension which should be compiled by installing sunpy. If for any reason this build process fails, you will not be ableto use the C-API rotate code, but will be able to still use all the functionality of map.

If this happens you will encounter the following warning upon using the rotate method

rot_map = mymap.rotate(10)sunpy/map/map.py:829: Warning: The C extension sunpy.image.Crotate is not installed, falling back to the interpolation=’spline’ of order=3

warnings.warn("The C extension sunpy.image.Crotate is not installed, falling back to the interpolation=’spline’ of order=3" ,Warning)

What happens is, because the C-API extension is not found, the rotate() function defaults to the spline interpolationmethod of order 3 which is implemented in scipy.

To fix the C-API you should try and reinstall SunPy, if this still fails please ask the mailing list for assistance.

10.2 Obtaining sunpy version

To find out your sunpy version number, import it and print the __version__ attribute:

import sunpysunpy.__version__

75


10.3 System Info

To quickly collect information on your system, you can use our convienience function system_info which you canrun through:

import sunpysunpy.util.system_info()

The output should look something like:

==========================================================SunPy Installation Information

Sunday, 18. November 2012 11:06PM UT==========================================================

###########General###########OS: Mac OS X 10.8.2 (i386)Python: 2.7.3 (64bit)

####################Required libraries####################SunPy: 0.1NumPy: 1.6.2SciPy: 0.10.1Matplotlib: 1.2.xPyFITS: 3.0.8pandas: 0.8.1

#######################Recommended libraries#######################beautifulsoup4: 4.1.1PyQt: 4.9.4SUDS: 0.4’

This information is especially useful if you are running into a bug and need help.

10.4 sunpy install location

You can find what directory sunpy is installed in by importing it and printing the __file__ attribute:

import sunpysunpy.__file__

10.5 .sunpy directory location

Each user should have a .sunpy/ directory which should contain a sunpyrc file. To locate your .sunpy/ directory, usesunpy.print_config():

76 Chapter 10. Troubleshooting


import sunpy as sunsun.print_config()

The output should look something like:

FILES USED:sunpy/data/sunpyrc

CONFIGURATION:[general]working_dir = /Users/schriste/sunpy

[downloads]download_dir = /Users/schriste/sunpy/data

On unix-like systems, this directory is generally located in your HOME directory. On windows, it is in your documentsand settings directory by default.

If you would like to use a different configuration directory, you can do so by specifying the location in yourSUNPY_CONFIGDIR environment variable.

10.6 Report a problem

If you are having a problem with sunpy, search the mailing lists first: it is possible that someone else has already runinto your problem.

If not, please provide the following information in your e-mail to the mailing list:

• your operating system; (Linux/UNIX users: post the output of uname -a)

• sunpy version:

import sunpysunpy.util.system_info()

• how you obtained sunpy.

• any customizations to your sunpyrc file (see Customizing sunpy).

• Please try to provide a minimal, standalone Python script that demonstrates the problem. This is the criticalstep. If you can’t post a piece of code that we can run and reproduce your error, the chances of getting helpare significantly diminished. Very often, the mere act of trying to minimize your code to the smallest bit thatproduces the error will help you find a bug in your code that is causing the problem.

You will likely get a faster response writing to the mailing list than filing a bug in the bug tracker. If your problem hasbeen determined to be a bug and can not be quickly solved, the issues may be filed a bug in the tracker so the issuedoesn’t get lost.

10.6. Report a problem 77

http://matplotlib.sourceforge.net/faq/environment_variables_faq.html#envvar-HOME

http://groups.google.com/forum/#!forum/sunpy

http://github.com/sunpy/sunpy/issues


78 Chapter 10. Troubleshooting

Part III

Acknowledging SunPy

79


If you have used SunPy in your scientific work we would appreciate it if you would acknowledge it.

81


82

CHAPTER 11

Publications

For a publication we recommend the following line be added

This research has made use of SunPy, an open-source and free community-developed solar data analysispackage written in Python (citation).

where the citation is to the SunPy paper presented at SciPy 2013 Scipy 2013 (Bib reference). The paper is open access.If the journal allows please also include a link to sunpy.org. If you have the time please email us to let us know aboutyour paper as we maintain a public list of papers Zotero.

83

http://conference.scipy.org/proceedings/scipy2013/mumford.html

http://conference.scipy.org/proceedings/scipy2013/

http://conference.scipy.org/proceedings/scipy2013/bib/mumford.bib

http://conference.scipy.org/proceedings/scipy2013/pdfs/mumford.pdf

mailto:[email protected]

https://www.zotero.org/groups/sunpy_-_python_for_solar_physicists

https://www.zotero.org


84 Chapter 11. Publications

CHAPTER 12

Posters/Presentations

If you are giving a talk or presenting a poster which used SunPy and would like to acknowledge it please include theSunpy logo on the title or conclusion slide.

Other versions of the logo are also available.

85

https://github.com/sunpy/sunpy/tree/documentation_sprint/doc/logo


86 Chapter 12. Posters/Presentations

Part IV

API Reference

87


This document is the API reference for SunPy, it will eventually be complete. Each submodule should be listed belowand document every public facing method. Any change to this API should result in a major version change.

89


90

CHAPTER 13

SunPy

13.1 sunpy Module

13.1.1 SunPy

An open-source Python library for Solar Physics data analysis.

Web Links

Homepage: http://www.sunpy.org Documentation: http://sunpy.readthedocs.org/en/latest/index.html

Available subpackages

cmSolar Physics specific color maps.

imageGeneric methods to work with image data or maps.

instrsubpackages spcific to missions or instruments.

lightcurvesubpackage for working with 1D data sets like lightcurves.

mapsubpackage for working with 2D and 3D data sets of images or sequences of images.

netRoutines for obtaining data from the internet.

databaseStore solar physics data (from FITS files or via the net package) in a database.

spectrasubpackage for working with 2D spectra datatypes

sunContains astronomical and physical constants.

91

http://www.sunpy.org

http://sunpy.readthedocs.org/en/latest/index.html


timeContains time related constants and methods.

wcsThe WCS package provides functions to parse a World Coordinate System(WCS) coordinates for solar imagesas well as convert between various solar coordinate systems.

13.1.2 Functions

load_config() Read the sunpyrc configuration file.print_config() Print current configuration options

load_config

sunpy.util.config.load_config()Read the sunpyrc configuration file. If one does not exists in the user’s home directory then read in the defaultsfrom module

print_config

sunpy.util.config.print_config()Print current configuration options

92 Chapter 13. SunPy

CHAPTER 14

SunPy cm

14.1 sunpy.cm Module

14.1.1 Functions

get_cmap([name]) Get a colormap.show_colormaps() Displays a plot of the custom color maps supported in SunPy.

get_cmap

sunpy.cm.cm.get_cmap(name=’sdoaia94’)Get a colormap.

Parametersname : string

The name of a color map.

Returnsvalue : matplotlib colormap

References

http://matplotlib.sourceforge.net/api/cm_api.html

Examples

>>> import sunpy.cm as cm>>> colormap = cm.get_cmap(name = ’sdoaia94’)

93

http://matplotlib.sourceforge.net/api/cm_api.html


show_colormaps

sunpy.cm.cm.show_colormaps()Displays a plot of the custom color maps supported in SunPy.

ParametersNone : none

ReturnsNone : none

Examples

>>> import sunpy.cm as cm>>> cm.show_colormaps()

94 Chapter 14. SunPy cm

CHAPTER 15

SunPy coords

15.1 sunpy.coords Module

15.1.1 Functions

diff_rot(ddays, latitude[, rot_type, frame_time]) This function computes the change in longitude over days in degrees.

diff_rot

sunpy.coords.util.diff_rot(ddays, latitude, rot_type=’howard’, frame_time=’sidereal’)This function computes the change in longitude over days in degrees.

Parametersddays: float or timedelta :

Number of days to rotate over, or timedelta object.

latitude: float or array-like :

heliographic coordinate latitude in Degrees.

rot_type: {‘howard’ | ‘snodgrass’ | ‘allen’} :

howard: Use values for small magnetic features from Howard et al. snodgrass: UseValues from Snodgrass et. al allen: Use values from Allen, Astrophysical Quantities,and simplier equation.

frame_time: {‘sidereal’ | ‘synodic’} :

Choose ‘type of day’ time reference frame.

Returnslongditude_delta: ndarray :

The change in longitude over days (units=degrees)

Notes

•IDL code equavalent: http://hesperia.gsfc.nasa.gov/ssw/gen/idl/solar/diff_rot.pro

95

http://hesperia.gsfc.nasa.gov/ssw/gen/idl/solar/diff_rot.pro


•Howard rotation: http://adsabs.harvard.edu/abs/1990SoPh..130..295H

•A review of rotation parameters (including Snodgrass values):http://link.springer.com/article/10.1023%2FA%3A1005226402796

Examples

Default rotation calculation over two days at 30 degrees latitude:

>>> rotation = diff_rot(2, 30)

Default rotation over two days for a number of latitudes:

>>> rotation = diff_rot(2, np.linspace(-70, 70, 20))

With rotation type ‘allen’:

>>> rotation = diff_rot(2, np.linspace(-70, 70, 20), ’allen’)

96 Chapter 15. SunPy coords

http://adsabs.harvard.edu/abs/1990SoPh..130..295H

http://link.springer.com/article/10.1023%2FA%3A1005226402796

CHAPTER 16

Database

16.1 Overview

The database package exports the following classes and exceptions:

classes

• Database

exceptions

• EntryAlreadyAddedError

• NoSuchEntryError

• EntryAlreadyStarredError

• EntryAlreadyUnstarredError

• EntryNotFoundError

• TagAlreadyAssignedError

• NoSuchTagError

• NonRemovableTagError

functions

• disable_undo

16.2 Package documentation

16.2.1 The Database class

class sunpy.database.Database(url[, CacheClass[, cache_size[, default_waveunit ]]])

97


Parametersurl : str

A URL describing the database. This value is simply passed tosqlalchemy.create_engine()

CacheClass : sunpy.database.caching.BaseCache

A concrete cache implementation of the abstract class BaseCache. Builtin sup-ported values for this parameters are sunpy.database.caching.LRUCacheand sunpy.database.caching.LFUCache. The default value issunpy.database.caching.LRUCache.

cache_size : int

The maximum number of database entries, default is no limit.

default_waveunit : str, optional

The wavelength unit that will be used if an entry is added to the database butits wavelength unit cannot be found (either in the file or the VSO query re-sult block, depending on the way the entry was added). If None (the default),attempting to add an entry without knowing the wavelength unit results in asunpy.database.WaveunitNotFoundError.

Attributes

16.2.2 Exceptions

class sunpy.database.EntryAlreadyAddedError(database_entry)This exception is raised if a database entry is attempted to be added to the database although it was alreadysaved in it.

class sunpy.database.EntryAlreadyStarredError(database_entry)This exception is raised if a database entry is marked as starred using Database.star() although it was alreadystarred before this operation.

class sunpy.database.EntryAlreadyUnstarredError(database_entry)This exception is raised if the star mark from a database entry is attempted to be removed although the entry isnot starred.

class sunpy.database.EntryNotFoundError(entry_id)This exception is raised if a database entry cannot be found by its unique ID.

class sunpy.database.TagAlreadyAssignedError(database_entry, tag_name)This exception is raised if it is attempted to assign a tag to a database entry but the database entry already hasthis tag assigned.

class sunpy.database.NoSuchTagError(tag_name)This exception is raised if a tag cannot be found in a database by its name.

98 Chapter 16. Database


http://docs.python.org/library/constants.html#None


16.3 Modules

16.3.1 tables

class sunpy.database.tables.DatabaseEntryThe class DatabaseEntry represents the main table of the database and each instance represents one recordthat can be saved in the database.

Parametersid : int

A unique ID number. By default it is None, but automatically set to the maximumnumber plus one when this entry is added to the database.

source : string

The source is the name of an observatory or the name of a network of observatories.

provider : string

The name of the server which provides the retrieved data.

physobs : string

A physical observable identifier used by VSO.

fileid : string

The file ID is a string defined by the data provider that should point to a specific dataproduct. The association of fileid to the specific data may change sometime, if the fileidalways points to the latest calibrated data.

observation_time_start : datetime

The date and time when the observation of the data started.

observation_time_end : datetime

The date and time when the observation of the data ended.

instrument : string

The instrument which was used to observe the data.

size : float

The size of the data in kilobytes.

wavemin : float

The value of the measured wave length.

wavemax : float

This is the same value as wavemin. The value is stored twice, because eachsuds.sudsobject.QueryResponseBlock which is used by the vso package containsboth these values.

path : string

A local file path where the according FITS file is saved.

download_time : datetime

The date and time when the files connected to a query have been downloaded. Note:this is not the date and time when this entry has been added to a database!

16.3. Modules 99


starred : bool

Entries can be starred to mark them. By default, this value is False.

fits_header_entries : list

A list of FitsHeaderEntry instances.

tags : list

A list of Tag instances. Use sunpy.database.Database.tag to add a new tag or multipletags to a specific entry.

class sunpy.database.tables.FitsHeaderEntry(key, value)

class sunpy.database.tables.FitsKeyComment(key, value)

class sunpy.database.tables.Tag(name)

class sunpy.database.tables.JSONDump(dump)

utility functions

sunpy.database.tables.entries_from_query_result(qr, default_waveunit=None)Use a query response returned from sunpy.net.vso.VSOClient.query() orsunpy.net.vso.VSOClient.query_legacy() to generate instances of DatabaseEntry. Return an iter-ator over those instances.

Parametersqr : sunpy.net.vso.vso.QueryResponse

The query response from which to build the database entries.


See sunpy.database.DatabaseEntry.from_query_result_block().

Examples

>>> from sunpy.net import vso>>> from sunpy.database import entries_from_query_result>>> client = vso.VSOClient()>>> qr = client.query(... vso.attrs.Time(’2001/1/1’, ’2001/1/2’),... vso.attrs.Instrument(’eit’))>>> entries = entries_from_query_result(qr)>>> entry = entries.next()>>> entry.source’SOHO’>>> entry.provider’SDAC’>>> entry.physobs’intensity’>>> entry.fileid’/archive/soho/private/data/processed/eit/lz/2001/01/efz20010101.010014’>>> entry.observation_time_start, entry.observation_time_end



(datetime.datetime(2001, 1, 1, 1, 0, 14), datetime.datetime(2001, 1, 1, 1, 0, 21))>>> entry.instrument’EIT’>>> entry.size2059.0>>> entry.wavemin, entry.wavemax(17.1, 17.1)

sunpy.database.tables.entries_from_file(file, default_waveunit=None)Use the headers of a FITS file to generate an iterator of sunpy.database.tables.DatabaseEntry instances.Gathered information will be saved in the attribute fits_header_entries. If the key INSTRUME,WAVELNTH or DATE-OBS / DATE_OBS is available, the attribute instrument, wavemin and wavemax orobservation_time_start is set, respectively. If the wavelength unit can be read, the values of wavemin andwavemax are converted to nm (nanometres). The value of the file parameter is used to set the attribute path ofeach generated database entry.

Parametersfile : str or file-like object

Either a path pointing to a FITS file or a an opened file-like object. If an opened fileobject, its mode must be one of the following rb, rb+, or ab+.


The wavelength unit that is used for a header if it cannot be found.

Raisessunpy.database.WaveunitNotFoundError :

If default_waveunit is not given and the wavelength unit cannot be found in one ofthe FITS headers

sunpy.WaveunitNotConvertibleError :

If a wavelength unit could be found but cannot be used to create an instance of the typeastropy.units.Unit. This can be the case for example if a FITS header has the keyWAVEUNIT with the value nonsense.

Examples

>>> entries = list(entries_from_file(sunpy.data.sample.SWAP_LEVEL1_IMAGE))>>> len(entries)1>>> entry = entries.pop()>>> entry.instrument’SWAP’>>> entry.observation_time_start, entry.observation_time_end(datetime.datetime(2012, 1, 1, 0, 16, 7, 836000), None)>>> entry.wavemin, entry.wavemax(17.400000000000002, 17.400000000000002)>>> len(entry.fits_header_entries)112

sunpy.database.tables.entries_from_dir(fitsdir, recursive=False, pattern=’*’, de-fault_waveunit=None)

Search the given directory for FITS files and use the corresponding FITS headers to generate instances ofDatabaseEntry. FITS files are detected by reading the content of each file, the pattern argument may be usedto avoid reading entire directories if one knows that all FITS files have the same filename extension.

16.3. Modules 101

http://docs.python.org/library/functions.html#file


Parametersfitsdir : string

The directory where to look for FITS files.

recursive : bool, optional

If True, the given directory will be searched recursively. Otherwise, only the given di-rectory and no subdirectories are searched. The default is False, i.e. the given directoryis not searched recursively.

pattern : string, optional

The pattern can be used to filter the list of filenames before the files are attemptedto be read. The default is to collect all files. This value is passed to the functionfnmatch.filter(), see its documentation for more information on the supported syn-tax.


See sunpy.database.tables.DatabaseEntry.add_fits_header_entries_from_file().

Returnsgenerator of (DatabaseEntry, str) pairs :

A generator where each item is a tuple consisting of a DatabaseEntry instance and theabsolute path to the filename which was used to make the database entry.

Examples

>>> from pprint import pprint>>> from sunpy.data.test import rootdir as fitsdir>>> entries = list(entries_from_dir(fitsdir))>>> len(entries)2>>> # and now search ‘fitsdir‘ recursive>>> entries = list(entries_from_dir(fitsdir, True))>>> len(entries)15>>> # print the first 5 items of the FITS header of the first found file>>> first_entry, filename = entries[0]>>> pprint(first_entry.fits_header_entries[:5])[<FitsHeaderEntry(id None, key ’SIMPLE’, value True)>,<FitsHeaderEntry(id None, key ’BITPIX’, value -64)>,<FitsHeaderEntry(id None, key ’NAXIS’, value 2)>,<FitsHeaderEntry(id None, key ’NAXIS1’, value 128)>,<FitsHeaderEntry(id None, key ’NAXIS2’, value 128)>]

sunpy.database.tables.display_entries(database_entries, columns)Generate a table to display the database entries.

Parametersdatabase_entries : iterable of DatabaseEntry instances

The database entries will be the rows in the resulting table.

columns : iterable of str

The columns that will be displayed in the resulting table. Possible values for the stringsare all attributes of DatabaseEntry.


http://docs.python.org/library/constants.html#False

http://docs.python.org/library/fnmatch.html#fnmatch.filter


Returnsstr :

A formatted table that can be printed on the console or written to a file.

Exceptions

class sunpy.database.tables.WaveunitNotFoundError(obj)This exception is raised if a wavelength unit cannot be found in a FITS header or in a VSO query result block.

class sunpy.database.tables.WaveunitNotConvertibleError(waveunit)This exception is raised if a wavelength cannot be converted to an astropy.units.Unit instance.

16.3.2 caching

class sunpy.database.caching.BaseCache(maxsize=inf)Bases: object

BaseCache is a class that saves and operates on an OrderedDict. It has a certain capacity, stored in the attributemaxsize. Whether this capacity is reached, can be checked by using the boolean property is_full. To imple-ment a custom cache, inherit from this class and override the methods __getitem__ and __setitem__. Call themethod sunpy.database.caching.BaseCache.callback() as soon as an item from the cache is removed.

Attributes

to_be_removed The item that will be removed on the next sunpy.database.caching.BaseCache.remove() call.is_full True if the number of items in the cache equals maxsize, False otherwise.

Methods

get(key[, default]) Return the corresponding value to key if key is in the cache, default otherwise.remove() Call this method to manually remove one item from the cache.callback(key, value) This method should be called (by convention) if an item is removed from the cache because it is full.__getitem__(key) abstract method: this method must be overwritten by inheriting__setitem__(key, value) abstract method: this method must be overwritten by inheriting

__getitem__(key)abstract method: this method must be overwritten by inheriting subclasses. It defines what happens if anitem from the cache is attempted to be accessed.

__setitem__(key, value)abstract method: this method must be overwritten by inheriting subclasses. It defines what happens if anew value should be assigned to the given key. If the given key does already exist in the cache or not mustbe checked by the person who implements this method.

callback(key, value)This method should be called (by convention) if an item is removed from the cache because it is full. Thepassed key and value are the ones that are removed. By default this method does nothing, but it can becustomized in a custom cache that inherits from this base class.

get(key, default=None)

16.3. Modules 103


Return the corresponding value to key if key is in the cache, default otherwise. This method has noside-effects, multiple calls with the same cache and the same passed key must always return the samevalue.

is_fullTrue if the number of items in the cache equals maxsize, False otherwise.

remove()Call this method to manually remove one item from the cache. Which item is removed, depends on theimplementation of the cache. After the item has been removed, the callback method is called.

to_be_removedThe item that will be removed on the next sunpy.database.caching.BaseCache.remove() call.

class sunpy.database.caching.LRUCache(maxsize=inf)Bases: sunpy.database.caching.BaseCache

Methods

get(key[, default]) Return the corresponding value to key if key is in the cache, default otherwise.remove() Remove the least recently used item.callback(key, value) This method should be called (by convention) if an item is removed from the cache because it is full.__getitem__(key) Returns the value which is associated to the given key and put it with its associated value to the end of this cache.__setitem__(key, value) If the key does already exist in the cache, move it to the end of this cache.

__getitem__(key)Returns the value which is associated to the given key and put it with its associated value to the end of thiscache.

RaisesKeyError :

If the key cannot be found in the cache.

__setitem__(key, value)If the key does already exist in the cache, move it to the end of this cache. Otherwise, set a new value andput it to the end of this cache. If the cache is full, remove the least recently used item before inserting thenew key-value pair.

remove()Remove the least recently used item.

to_be_removedReturn the least recently used key and its corresponding value as a tuple.

class sunpy.database.caching.LFUCache(maxsize=inf)Bases: sunpy.database.caching.BaseCache

Methods

get(key[, default]) Return the corresponding value to key if key is in the cache, default otherwise.remove() Remove the least frequently used item.callback(key, value) This method should be called (by convention) if an item is removed from the cache because it is full.__getitem__(key) Returns the value which is associated to the given key and increments the frequency counter of this key.

Continued on next page



Table 16.5 – continued from previous page__setitem__(key, value) Increment the frequency counter of the given key if it is already present in the cache, otherwise set it to 1.

__getitem__(key)Returns the value which is associated to the given key and increments the frequency counter of this key.

RaisesKeyError :

If the key cannot be found in the cache.

__setitem__(key, value)Increment the frequency counter of the given key if it is already present in the cache, otherwise set it to 1.If the cache is full, remove the least frequently used item before inserting the new key-value pair.

remove()Remove the least frequently used item.

to_be_removedReturns the key with the lowest times of access and its corresponding value as a tuple.

16.3.3 commands

class sunpy.database.commands.DatabaseOperationBases: object

This is the abstract main class for all database operations. To implement a new operation, inherit from this classand override the methods __call__ and undo. Both these methods get no parameters (except for self of course).The undo method is expected to do the exact opposite of the __call__ method, so that calling __call__ and undomultiple times in a row must not have any side-effects. This is not checked in any way, though.

class sunpy.database.commands.AddEntry(session, database_entry)Bases: sunpy.database.commands.DatabaseOperation

Add a new database entry to this session. It is not checked whether an equivalent entry is already saved in thesession; this has to be checked by the caller. The undo method removes the entry from the session again.

class sunpy.database.commands.RemoveEntry(session, entry)Bases: sunpy.database.commands.DatabaseOperation

Remove the given database entry from the session. If it cannot be removed, because it is not stored in the session,sunpy.database.NoSuchEntryError is raised. The undo method puts the database entry back into the sessionobject.

class sunpy.database.commands.EditEntry(database_entry, **kwargs)Bases: sunpy.database.commands.DatabaseOperation

Change the properties of the database entry. The given keyword arguments are used to set the attributes of theentry. The keys represent the attribute name and the values represent the new value of this attribute. Example:EditEntry(entry, foo=’bar’) will set the attribute foo of entry to the value ’bar’.

class sunpy.database.commands.CommandManagerBases: object

The CommandManager saves all executed and reverted commands to act as an undo-redo-manager. All ex-ecuted commands are saved in the list attribute undo_commands and all undone commands are saved in thelist attribute redo_commands. It is not recommended to alter these stacks directly; instead, use the methodspush_undo_command, pop_undo_command, push_redo_command, and pop_redo_command, respectively.

16.3. Modules 105


do(command)Execute the given command (a subclass of DatabaseOperation). Exceptions raised from the command arenot caught. The passed argument may also be an iterable of commands. In this case, every command ofthe iterable is executed and only one entry is saved in the undo history.

pop_redo_command()Remove the last command from the redo command stack and return it. If the command stack is empty,sunpy.database.commands.EmptyCommandStackError is raised.

pop_undo_command()Remove the last command from the undo command stack and return it. If the command stack is empty,sunpy.database.commands.EmptyCommandStackError is raised.

push_redo_command(command)Push the given command to the redo command stack.

push_undo_command(command)Push the given command to the undo command stack.

redo(n=1)Redo the last n commands which have been undone using the undo method. The default isto redo only the last command which has been undone using the undo method. If there is nocommand that can be redone because n is too big or because no command has been undone yet,sunpy.database.commands.EmptyCommandStackError is raised.

undo(n=1)Undo the last n commands. The default is to undo only the last command. If there is no com-mand that can be undone because n is too big or because no command has been executed yet,sunpy.database.commands.EmptyCommandStackError is raised.

Exceptions

class sunpy.database.commands.EmptyCommandStackErrorThis exception is raised if it is attempted to pop from a command stack even though it is empty.

class sunpy.database.commands.NoSuchEntryError(database_entry)This exception is raised if it is attempted to remove an entry even though it does not exist in the database.

class sunpy.database.commands.NonRemovableTagError(database_entry, tag)This exception is raised if it is attempted to remove a tag from a database entry even though it is not saved inthis entry.

16.3.4 attrs

class sunpy.database.attrs.Starred

class sunpy.database.attrs.Tag(tagname)

class sunpy.database.attrs.Path(value)

class sunpy.database.attrs.DownloadTime(start, end)

class sunpy.database.attrs.FitsHeaderEntry(key, value)


CHAPTER 17

SunPy image

17.1 sunpy.image Module

17.1.1 sunpy.image.rescale Module

Image resampling methods

Functions

resample(orig, dimensions[, method, center, ...]) Returns a new ndarray that has been resampled up or down Arbitrary resampling of source array to new dimension sizes.reshape_image_to_4d_superpixel(img, dimensions) Re-shape the two dimension input image into a a four dimensional array whose 1st and third dimensions express the number of original pixels in the x and y directions form one superpixel.

resample

sunpy.image.rescale.resample(orig, dimensions, method=’linear’, center=False, minusone=False)Returns a new ndarray that has been resampled up or down

Arbitrary resampling of source array to new dimension sizes. Currently only supports maintaining the samenumber of dimensions. To use 1-D arrays, first promote them to shape (x,1).

Uses the same parameters and creates the same co-ordinate lookup points as IDL”s congrid routine, whichapparently originally came from a VAX/VMS routine of the same name.

Parametersdimensions : tuple

Dimensions that new ndarray should have.

method : {‘neighbor’ | ‘nearest’ | ‘linear’ | ‘spline’}

Method to use for resampling interpolation.

•neighbor - Closest value from original data

•nearest and linear - Uses n x 1-D interpolations using scipy.interpolate.interp1d

•spline - Uses ndimage.map_coordinates

107


center : bool

If True, interpolation points are at the centers of the bins, otherwise points are at thefront edge of the bin.

minusone : bool

For inarray.shape = (i,j) & new dimensions = (x,y), if set to False inarray is resampledby factors of (i/x) * (j/y), otherwise inarray is resampled by(i-1)/(x-1) * (j-1)/(y-1) Thisprevents extrapolation one element beyond bounds of input array.

Returnsout : ndarray

A new ndarray which has been resampled to the desired dimensions.

References

http://www.scipy.org/Cookbook/Rebinning (Original source, 2011/11/19)

reshape_image_to_4d_superpixel

sunpy.image.rescale.reshape_image_to_4d_superpixel(img, dimensions)Re-shape the two dimension input image into a a four dimensional array whose 1st and third dimensions expressthe number of original pixels in the x and y directions form one superpixel. The reshaping makes it very easyto perform operations on super-pixels. Taken from http://mail.scipy.org/pipermail/numpy-discussion/2010-July/051760.html

108 Chapter 17. SunPy image

http://www.scipy.org/Cookbook/Rebinning

http://mail.scipy.org/pipermail/numpy-discussion/2010-July/051760.html


CHAPTER 18

SunPy Map

18.1 Overview

One of core classes in SunPy is a Map. A SunPy Map object is simply a spatially-aware data array, often an image. Inorder to make it easy to work with image data in SunPy, the Map object provides a number of methods for commonlyperformed operations.

2D map objects are subclasses of sunpy.map.MapBase and all Map objects are created using the Map factorysunpy.Map.

18.2 Creating Map Objects

SunPy Map objects are constructed using the special factory class Map:

>>> x = sunpy.Map(’file.fits’)

The result of a call to Map will be either a MapBase object, or a subclass of MapBase which either deals with a specifictype of data, e.g. AIAMap or LASCOMap, or if no instrument matches, a 2D map GenericMap.

The SunPy Map factory accepts a wide variety of inputs for creating maps:

* Preloaded tuples of (data, header) pairs

>>> mymap = sunpy.Map((data, header))

headers are some base of dict or OrderedDict,including SunPy’s FileHeader or Map Meta classes.

• data, header pairs, not in tuples

>>> mymap = sunpy.Map(data, header)

• File names

>>> mymap = sunpy.Map(’file1.fits’)

• All fits files in a directory by giving a directory

109

http://docs.python.org/library/stdtypes.html#dict


>>> mymap = sunpy.Map(’local_dir/sub_dir’)

• Some regex globs

>>> mymap = sunpy.Map(’eit_*.fits’)

• URLs

>>> mymap = sunpy.Map(url_str)

• Lists of any of the above

>>> mymap = sunpy.Map([’file1.fits’, ’file2.fits’, ’file3.fits’, ’directory1/’])

• Any mixture of the above not in a list

>>> mymap = sunpy.Map((data, header), data2, header2, ’file1.fits’, url_str, ’eit_*.fits’)

sunpy.map.Map.__new__()

class sunpy.map.Map(*args, **kwargs)Map factory class. Used to create a variety of Map objects. Valid map types are specified by registering themwith the factory.

18.3 Map Classes

There are a series of base map classes which are specalised for each instrument. These subclass GenericMap and thenregister with the Map factory class, which will direct instatiation of an instrument class if the correct parameters aremet.

18.3.1 Classes

CompositeMap(map1 [,map2,..]) A Composite Map class :Parameters: args : [sunpy.map | string] One or more map of filepaths ..GenericMap(data, header, **kwargs) A Generic spatially-aware 2D data array :Parameters: data : numpy.ndarray, list A 2d list or ndarray containing the map data header : dict A dictionary of the original image header tags ..Map(*args, **kwargs) Map factory class.MapCube(input) A series of spatially aligned Maps.MapMeta(adict) A class to hold meta data associated with a Map derivative.

CompositeMap

class sunpy.map.compositemap.CompositeMap(map1[, map2, ..])Bases: object

A Composite Map class

Parametersargs : [sunpy.map | string]

One or more map of filepaths

110 Chapter 18. SunPy Map


Examples

>>> import sunpy>>> sunpy.Map(sunpy.AIA_171_IMAGE, sunpy.RHESSI_IMAGE, composite=True)>>> comp_map = sunpy.Map(sunpy.AIA_171_IMAGE, sunpy.EIT_195_IMAGE,

composite=True)>>> comp_map.add_map(sunpy.Map(sunpy.RHESSI_IMAGE))>>> comp_map.peek()

Methods

add_map(map, zorder=None, alpha=1,levels=False)

Adds a map to the CompositeMap

remove_map(index) Removes and returns the map with the given indexlist_maps() Prints a list of the currently included mapsget_alpha(index=None) Gets the alpha-channel value for a layer in the

composite imageget_zorder(index=None) Gets the layering preference (z-order) for a map

within the composite.get_colors(index=None) Gets the colors for a map within the CompositeMap.get_norm(index=None) Gets the normalization for a map within the

CompositeMap.get_levels(index=None) Gets the list of contour levels for a map within the

CompositeMapset_norm(self, index, norm) Sets the norm for a layer in the composite imageset_levels(index, levels, percent=False) Sets the contour levels for a layer in the

CompositeMapset_colors(index=None, cm) Sets the color map for a layer in the CompositeMapset_alpha(index=None, alpha) Sets the alpha-channel value for a layer in the

CompositeMapset_zorder(index=None, zorder) Set the layering preference (z-order) for a map

within the CompositeMapplot(figure=None, overlays=None,draw_limb=False, gamma=1.0,draw_grid=False, colorbar=True,basic_plot=False,title=”SunPy Plot”,matplot_args) Plots the composite map object using matplotlib

Methods Summary

add_map(amap[, zorder, alpha, levels]) Adds a map to the CompositeMap :Parameters: map : sunpy.map Map instance to be added zorder : int The index to use when determining where the map should lie along the z-axis; maps with higher z-orders appear above maps with lower z-orders.draw_grid([index, axes, grid_spacing]) Draws a grid over the surface of the Sun :Parameters: index: integer : Index to determine which map to use to draw grid.draw_limb([index, axes]) Draws a circle representing the solar limb :Parameters: index: integer : Map index to use to plot limb.get_alpha([index]) Gets the alpha-channel value for a layer in the composite imageget_colors([index]) Gets the colors for a map within the compositemap.get_levels([index]) Gets the list of contour levels for a map within the composite.get_map(index) Returns the map with given indexget_norm([index]) Gets the normalization for a map within the composite.get_zorder([index]) Gets the layering preference (z-order) for a map within the composite.

Continued on next page

18.3. Map Classes 111


Table 18.2 – continued from previous pagelist_maps() Prints a list of the currently included mapspeek([gamma, colorbar, basic_plot, ...]) Displays the map in a new figure :Parameters: gamma : float Gamma value to use for the color map colorbar : bool or int Whether to display a colorbar next to the plot.plot([axes, gamma, annotate, title]) Plots the composite map object using matplotlib :Parameters: axes: matplotlib.axes object or None : If provided the image will be plotted on the given axes.remove_map(index) Removes and returns the map with the given indexset_alpha(index, alpha) Sets the alpha-channel value for a layer in the composite imageset_colors(index, cm) Sets the color map for a layer in the composite imageset_levels(index, levels[, percent]) Sets the contour levels for a layer in the composite imageset_norm(index, norm) Sets the norm for a layer in the composite imageset_zorder(index, zorder) Set the layering preference (z-order) for a map within the composite.

Methods Documentation

add_map(amap, zorder=None, alpha=1, levels=False)Adds a map to the CompositeMap

Parametersmap : sunpy.map

Map instance to be added

zorder : int

The index to use when determining where the map should lie along the z-axis; mapswith higher z-orders appear above maps with lower z-orders.

alpha : float

Opacity at which the map should be displayed. An alpha value of 0 results in a fullytransparent image while an alpha value of 1 results in a fully opaque image. Valuesbetween result in semi- transparent images.

draw_grid(index=None, axes=None, grid_spacing=20)Draws a grid over the surface of the Sun

Parametersindex: integer :

Index to determine which map to use to draw grid.

axes: matplotlib.axes object or None :

Axes to plot limb on or None to use current axes.

grid_spacing: float :

Spacing (in degrees) for longitude and latitude grid.

Returnsmatplotlib.axes object :

draw_limb(index=None, axes=None)Draws a circle representing the solar limb

Parametersindex: integer :

Map index to use to plot limb.






get_alpha(index=None)Gets the alpha-channel value for a layer in the composite image

get_colors(index=None)Gets the colors for a map within the compositemap.

get_levels(index=None)Gets the list of contour levels for a map within the composite.

get_map(index)Returns the map with given index

get_norm(index=None)Gets the normalization for a map within the composite.

get_zorder(index=None)Gets the layering preference (z-order) for a map within the composite.

list_maps()Prints a list of the currently included maps

peek(gamma=None, colorbar=True, basic_plot=False, draw_limb=True, draw_grid=False, **mat-plot_args)

Displays the map in a new figure

Parametersgamma : float

Gamma value to use for the color map

colorbar : bool or int

Whether to display a colorbar next to the plot. If specified as an integer a colorbar isplotted for that index.

basic_plot : bool

If true, the data is plotted by itself at it’s natural scale; no title, labels, or axes are shown.

**matplot_args : dict

Matplotlib Any additional imshow arguments that should be used when plotting theimage.

plot(axes=None, gamma=None, annotate=True, title=’SunPy Composite Plot’, **matplot_args)Plots the composite map object using matplotlib

Parametersaxes: matplotlib.axes object or None :

If provided the image will be plotted on the given axes. Else the current matplotlib axeswill be used.

gamma : float


annotate : bool

If true, the data is plotted at it’s natural scale; with title and axis labels.





Returnsret : List

List of axes image or quad contour sets that have been plotted.

remove_map(index)Removes and returns the map with the given index

set_alpha(index, alpha)Sets the alpha-channel value for a layer in the composite image

set_colors(index, cm)Sets the color map for a layer in the composite image

set_levels(index, levels, percent=False)Sets the contour levels for a layer in the composite image

set_norm(index, norm)Sets the norm for a layer in the composite image

set_zorder(index, zorder)Set the layering preference (z-order) for a map within the composite.

GenericMap

class sunpy.map.mapbase.GenericMap(data, header, **kwargs)Bases: sunpy.map.mapbase.NDDataStandin

A Generic spatially-aware 2D data array

Parametersdata : numpy.ndarray, list

A 2d list or ndarray containing the map data

header : dict

A dictionary of the original image header tags

References

http://docs.scipy.org/doc/numpy/reference/arrays.classes.htmlhttp://docs.scipy.org/doc/numpy/user/basics.subclassing.htmlhttp://docs.scipy.org/doc/numpy/reference/ufuncs.htmlhttp://www.scipy.org/Subclasses

Examples

>>> aia = sunpy.make_map(sunpy.AIA_171_IMAGE)>>> aia.TAIAMap([[ 0.3125, 1. , -1.1875, ..., -0.625 , 0.5625, 0.5 ],[-0.0625, 0.1875, 0.375 , ..., 0.0625, 0.0625, -0.125 ],[-0.125 , -0.8125, -0.5 , ..., -0.3125, 0.5625, 0.4375],...,


http://docs.scipy.org/doc/numpy/reference/arrays.classes.html

http://docs.scipy.org/doc/numpy/user/basics.subclassing.html

http://docs.scipy.org/doc/numpy/reference/ufuncs.html

http://www.scipy.org/Subclasses


[ 0.625 , 0.625 , -0.125 , ..., 0.125 , -0.0625, 0.6875],[-0.625 , -0.625 , -0.625 , ..., 0.125 , -0.0625, 0.6875],[ 0. , 0. , -1.1875, ..., 0.125 , 0. , 0.6875]])>>> aia.units[’x’]’arcsec’>>> aia.peek()

Attributes Summary

carrington_longitude Carrington longitude (crln_obs)center X and Y coordinate of the center of the map in units.coordinate_system Coordinate system used for x and y axes (ctype1/2)date Image observation timedetector Detector namedsun The observer distance from the Sun.exposure_time Exposure time of the image in seconds.heliographic_latitude Heliographic latitude in degreesheliographic_longitude Heliographic longitude in degreesinstrument Instrument namemeasurement Measurement name, defaults to the wavelength of imagename Human-readable description of map-typenickname An abbreviated human-readable description of the map-type; part of the Helioviewer data modelobservatory Observatory or Telescope namereference_coordinate Reference point WCS axes in data units (crval1/2)reference_pixel Reference point axes in pixels (crpix1/2)rotation_anglersun_arcseconds Radius of the sun in arcsecondsrsun_meters Radius of the sun in metersscale Image scale along the x and y axes in units/pixel (cdelt1/2)units Image coordinate units along the x and y axes (cunit1/2).wavelength wavelength of the observationxrange Return the X range of the image in arcsec from edge to edge.yrange Return the Y range of the image in arcsec from edge to edge.

Methods Summary

data_to_pixel(value, dim) Convert pixel-center data coordinates to pixel valuesdraw_grid([axes, grid_spacing]) Draws a grid over the surface of the Sun :Parameters: axes: matplotlib.axes object or None : Axes to plot limb on or None to use current axes. : grid_spacing: float : Spacing (in degrees) for longitude and latitude grid.draw_limb([axes]) Draws a circle representing the solar limb :Parameters: axes: matplotlib.axes object or None : Axes to plot limb on or None to use current axes.peek([draw_limb, draw_grid, gamma, ...]) Displays the map in a new figure :Parameters: draw_limb : bool Whether the solar limb should be plotted.pixel_to_data([x, y]) Convert from pixel coordinates to data coordinates (e.g.plot([gamma, annotate, axes]) Plots the map object using matplotlib, in a method equivalent to plt.imshow() using nearest neighbour interpolation.resample(dimensions[, method]) Returns a new Map that has been resampled up or down Arbitrary resampling of the Map to new dimension sizes.rotate(angle[, scale, rotation_center, ...]) Returns a new rotated, rescaled and shifted map.save(filepath[, filetype]) Saves the SunPy Map object to a file.submap(range_a, range_b[, units]) Returns a submap of the map with the specified range :Parameters: range_a : list The range of the Map to select across either the x axis.superpixel(dimensions[, method]) Returns a new map consisting of superpixels formed from the original data.



Attributes Documentation

carrington_longitudeCarrington longitude (crln_obs)

centerX and Y coordinate of the center of the map in units. Usually represents the offset between the center ofthe Sun and the center of the map.

coordinate_systemCoordinate system used for x and y axes (ctype1/2)

dateImage observation time

detectorDetector name

dsunThe observer distance from the Sun.

exposure_timeExposure time of the image in seconds.

heliographic_latitudeHeliographic latitude in degrees

heliographic_longitudeHeliographic longitude in degrees

instrumentInstrument name

measurementMeasurement name, defaults to the wavelength of image

nameHuman-readable description of map-type

nicknameAn abbreviated human-readable description of the map-type; part of the Helioviewer data model

observatoryObservatory or Telescope name

reference_coordinateReference point WCS axes in data units (crval1/2)

reference_pixelReference point axes in pixels (crpix1/2)

rotation_angle

rsun_arcsecondsRadius of the sun in arcseconds

rsun_metersRadius of the sun in meters

scaleImage scale along the x and y axes in units/pixel (cdelt1/2)



unitsImage coordinate units along the x and y axes (cunit1/2).

wavelengthwavelength of the observation

xrangeReturn the X range of the image in arcsec from edge to edge.

yrangeReturn the Y range of the image in arcsec from edge to edge.


data_to_pixel(value, dim)Convert pixel-center data coordinates to pixel values

draw_grid(axes=None, grid_spacing=20, **kwargs)Draws a grid over the surface of the Sun


Axes to plot limb on or None to use current axes. :

grid_spacing: float :

Spacing (in degrees) for longitude and latitude grid.


Notes

keyword arguments are passed onto matplotlib.pyplot.plot

draw_limb(axes=None, **kwargs)Draws a circle representing the solar limb




Notes

keyword arguments are passed onto the Circle Patch, see: http://matplotlib.org/api/artist_api.html#matplotlib.patches.Patchhttp://matplotlib.org/api/artist_api.html#matplotlib.patches.Circle

peek(draw_limb=True, draw_grid=False, gamma=None, colorbar=True, basic_plot=False, **mat-plot_args)

Displays the map in a new figure

Parametersdraw_limb : bool


http://matplotlib.org/api/artist_api.html#matplotlib.patches.Patch

http://matplotlib.org/api/artist_api.html#matplotlib.patches.Circle


Whether the solar limb should be plotted.

draw_grid : bool or number

Whether solar meridians and parallels are plotted. If float then sets degree differencebetween parallels and meridians.

gamma : float


colorbar : bool

Whether to display a colorbar next to the plot

basic_plot : bool

If true, the data is plotted by itself at it’s natural scale; no title, labels, or axes are shown.



pixel_to_data(x=None, y=None)Convert from pixel coordinates to data coordinates (e.g. arcsec)

plot(gamma=None, annotate=True, axes=None, **imshow_args)Plots the map object using matplotlib, in a method equivalent to plt.imshow() using nearest neighbourinterpolation.

Parametersgamma : float


annotate : bool




**imshow_args : dict

Any additional imshow arguments that should be used when plotting the image.

Examples

#Simple Plot with color bar plt.figure() aiamap.plot() plt.colorbar()

#Add a limb line and grid aia.plot() aia.draw_limb() aia.draw_grid()

resample(dimensions, method=’linear’)Returns a new Map that has been resampled up or down

Arbitrary resampling of the Map to new dimension sizes.

Uses the same parameters and creates the same co-ordinate lookup points as IDL”s congrid routine, whichapparently originally came from a VAX/VMS routine of the same name.




Dimensions that new Map should have. Note: the first argument corresponds to the ‘x’axis and the second argument corresponds to the ‘y’ axis.

method : {‘neighbor’ | ‘nearest’ | ‘linear’ | ‘spline’}

Method to use for resampling interpolation.

•neighbor - Closest value from original data

•nearest and linear - Uses n x 1-D interpolations using scipy.interpolate.interp1d

•spline - Uses ndimage.map_coordinates

Returnsout : Map

A new Map which has been resampled to the desired dimensions.

References

http://www.scipy.org/Cookbook/Rebinning (Original source, 2011/11/19)

rotate(angle, scale=1.0, rotation_center=None, recenter=True, missing=0.0, interpolation=’bicubic’,interp_param=-0.5)

Returns a new rotated, rescaled and shifted map.

Parametersangle: float :

The angle to rotate the image by (radians)

scale: float :

A scale factor for the image, default is no scaling

rotation_center: tuple :

The point in the image to rotate around (Axis of rotation). Default: center of the array

recenter: bool, or array-like :

Move the centroid (axis of rotation) to the center of the array or recenter coords. Default:True, recenter to the center of the array.

missing: float :

The numerical value to fill any missing points after rotation. Default: 0.0

interpolation: {‘nearest’ | ‘bilinear’ | ‘spline’ | ‘bicubic’} :

Interpolation method to use in the transform. Spline uses thescipy.ndimage.interpolation.affline_transform routine. nearest, bilinear and bicu-bic all replicate the IDL rot() function. Default: ‘bicubic’

interp_par: Int or Float :

Optional parameter for controlling the interpolation. Spline interpolation requires aninteger value between 1 and 5 for the degree of the spline fit. Default: 3 BiCubicinterpolation requires a flaot value between -1 and 0. Default: 0.5 Other interpolationoptions ingore the argument.


http://www.scipy.org/Cookbook/Rebinning


ReturnsNew rotated, rescaled, translated map :

Notes

Apart from interpolation=’spline’ all other options use a compiled C-API extension. If for some reasonthis is not compiled correctly this routine will fall back upon the scipy implementation of order = 3. Formore infomation see: http://sunpy.readthedocs.org/en/latest/guide/troubleshooting.html#crotate-warning

save(filepath, filetype=’auto’, **kwargs)Saves the SunPy Map object to a file.

Currently SunPy can only save files in the FITS format. In the future support will be added for saving toother formats.

Parametersfilepath : string

Location to save file to.

filetype : string

‘auto’ or any supported file extension

submap(range_a, range_b, units=’data’)Returns a submap of the map with the specified range

Parametersrange_a : list

The range of the Map to select across either the x axis.

range_b : list

The range of the Map to select across either the y axis.

units : {‘data’ | ‘pixels’}, optional

The units for the supplied ranges.

Returnsout : Map

A new map instance is returned representing to specified sub-region

Examples

>>> aia.submap([-5,5],[-5,5])AIAMap([[ 341.3125, 266.5 , 329.375 , 330.5625, 298.875 ],[ 347.1875, 273.4375, 247.4375, 303.5 , 305.3125],[ 322.8125, 302.3125, 298.125 , 299. , 261.5 ],[ 334.875 , 289.75 , 269.25 , 256.375 , 242.3125],[ 273.125 , 241.75 , 248.8125, 263.0625, 249.0625]])

>>> aia.submap([0,5],[0,5], units=’pixels’)AIAMap([[ 0.3125, -0.0625, -0.125 , 0. , -0.375 ],[ 1. , 0.1875, -0.8125, 0.125 , 0.3125],[-1.1875, 0.375 , -0.5 , 0.25 , -0.4375],[-0.6875, -0.3125, 0.8125, 0.0625, 0.1875],[-0.875 , 0.25 , 0.1875, 0. , -0.6875]])


http://sunpy.readthedocs.org/en/latest/guide/troubleshooting.html#crotate-warning


superpixel(dimensions, method=’sum’)Returns a new map consisting of superpixels formed from the original data. Useful for increasing signalto noise ratio in images.


One superpixel in the new map is equal to (dimension[0], dimension[1]) pixels of theoriginal map Note: the first argument corresponds to the ‘x’ axis and the second argu-ment corresponds to the ‘y’ axis.

method : {‘sum’ | ‘average’}

What each superpixel represents compared to the original data

•sum - add up the original data

•average - average the sum over the number of original pixels

Returnsout : Map

A new Map which has superpixels of the required size.

References


Map

class sunpy.map.map_factory.Map(*args, **kwargs)Bases: sunpy.util.datatype_factory_base.RegisteredFactoryBase

Map factory class. Used to create a variety of Map objects. Valid map types are specified by registering themwith the factory.

MapCube

class sunpy.map.mapcube.MapCube(input)Bases: object

A series of spatially aligned Maps.

Parametersargs : {List}

A list of Map instances

sortby : {“date”, None}

Method by which the MapCube should be sorted along the z-axis.

derotate : {None}

Apply a derotation to the data (Not Implemented)

coalign : {None}

Apply fine coalignment to the data (Not Implemented)




Examples

>>> mapcube = sunpy.Map(’images/’, mapcube=True)>>> mapcube[0].plot()>>> mapcube[3].reference_pixel[’x’]2050.6599120000001

Methods Summary

coalign([method]) Fine coalign the dataplot([gamma, annotate, axes, controls, ...]) A animation plotting routine that animates each element in the MapCube :Parameters: gamma: float : Gamma value to use for the color map annotate: bool : If true, the data is plotted at it’s natural scale; with title and axis labels.


coalign(method=’diff’)Fine coalign the data

plot(gamma=None, annotate=True, axes=None, controls=False, interval=200, resample=False, color-bar=False, ani_args={}, **imshow_args)

A animation plotting routine that animates each element in the MapCube

Parametersgamma: float :


annotate: bool :




controls: bool :

Adds play / pause button to the animation

interval: int :

Frame display time in ms.

resample: list or False :

Draws the map at a lower resolution to increase the speed of animation. Specify a listas a fraction i.e. [0.25, 0.25] to plot at 1/4 resolution.

colorbar: bool :

Draw a colorbar on the plot.

ani_args : dict

Passed to sunpy.util.plotting.ControlFuncAnimation

imshow_args: dict :

Any additional imshow arguments that should be used when plotting the image.



Examples

>>> cube = sunpy.Map(files, cube=True)>>> ani = cube.plot(colorbar=True)>>> plt.show()

Plot the map at 1/2 original resolution

>>> cube = sunpy.Map(files, cube=True)>>> ani = cube.plot(resample=[0.5, 0.5], colorbar=True)>>> plt.show()

Save an animation of the MapCube

>>> cube = sunpy.Map(res, cube=True)

>>> ani = cube.plot(controls=False)

>>> Writer = animation.writers[’ffmpeg’]>>> writer = Writer(fps=10, metadata=dict(artist=’SunPy’), bitrate=1800)

>>> ani.save(’mapcube_animation.mp4’, writer=writer)

MapMeta

class sunpy.map.header.MapMeta(adict)Bases: collections.OrderedDict

A class to hold meta data associated with a Map derivative.

This class handles everything a lower case. This allows case insensitive indexing.

Methods Summary

get(key[, default]) Overide .get() indexinghas_key(key) Overide .has_key() to perform case-insensitivelypop(key[, default]) Overide .pop() to perform case-insensitivelysetdefault(key[, default]) Overide .setdefault() to perform case-insensitivelyupdate(d2) Overide .update() to perform case-insensitively


get(key, default=None)Overide .get() indexing

has_key(key)Overide .has_key() to perform case-insensitively

pop(key, default=None)Overide .pop() to perform case-insensitively

setdefault(key, default=None)Overide .setdefault() to perform case-insensitively


http://docs.python.org/library/collections.html#collections.OrderedDict


update(d2)Overide .update() to perform case-insensitively

18.3.2 Class Inheritance Diagram

sunpy.map.header.MapMetacollections.OrderedDict

sunpy.map.map_factory.Mapsunpy.util.datatype_factory_base.RegisteredFactoryBase

sunpy.map.compositemap.CompositeMap

sunpy.map.mapbase.NDDataStandin sunpy.map.mapbase.GenericMap

sunpy.map.mapcube.MapCube

18.4 Writing a new Map Class

Map classes can be registered with the Map factory, even if the new class is not officially part of SunPy. This is goodfor prototyping new instruments. For example, to add a Map type for a future instrument, consider the code skeleton:

import sunpy

class FutureMap(sunpy.GenericMap):

def __init__(self, data, header, **kwargs):

GenericMap.__init__(self, data, header, **kwargs)

# Any Future Instrument specific keyword manipulation

# Specify a classmethod that determines if the data-header pair matches# the new instrument@classmethoddef is_datasource_for(cls, data, header, **kwargs):

"""Determines if header corresponds to an AIA image"""return header.get(’instrume’, ’’).startswith(’FUTURESCOPE’)

Then, to be able to instantiate a FutureMap using the Map() factory, one must register the FutureMap type with thefactory

sunpy.Map.register(FutureMap, FutureMap.is_datasource_for)

If this line is placed correctly, for example in your subpackages __init__.py, it can be guaranteed that the FutureMapis always accessible when your package is imported.


Part V

Reporting Bugs

125


All bugs are kept track of on the GitHub issue tracker. If you run into any unexpected behavior or have a questionplease send an email or add an issue directly to the issue tracker.

127

https://github.com/sunpy/sunpy/issues



128

Part VI

Developer’s Guide

129

CHAPTER 19

Developer’s Guide Overview

This article describes the guidelines to be followed by developers working on SunPy. You if you are thinking ofcontributing to SunPy please read the following carefully.

131


132 Chapter 19. Developer’s Guide Overview

CHAPTER 20

Version Control

Source-code for SunPy is managed using Git, a Distributed Version Control system. Code branches are hosted onGitHub.com, a free project hosting website for Open-Source software.

20.1 Creating Your Own Repo

Overview

Each person contributing to SunPy should create their own code repository on GitHub by forking the master repositoryor repo.eas All development is then done on that fork, using topic branches to isolate work on different features. Newcontributers can then initiate pull requests to have their code incorporated into the SunPy master repository. Regularcontributers can become members of the SunPy team on GitHub. Code will be reviewed by regular contributors andcomments will usually be provided before code is accepted.

Getting Started

Creating your own repo on GitHub is easy to do. If you followed the SunPy installation instructions you shouldalready have git installed. Go ahead and create a free account on create an account on GitHub. Github has some greatresources to help. Here is a quick overview of the process.

Adding an SSH key to GitHub

Next, you need to tell GitHub who you are. In order to push any code to GitHub you need to create a public SSH keyand associate it with your GitHub account. For instructions on how this is done, see the article on GitHub on Settingup git under “Set Up SSH Keys”. You only need to do this once, although if you plan to work from multiple computersyou will need to go through the process for each computer you wish to work on. Once you have created your accountand associated a public SSH key it, you are ready to go.

Using HTTPS

If you do not fancy using SSH you can access GitHub using HTTP/HTTPS. A few things to note. Using HTTP onlyallows cloning of public repositories, while HTTPS allows cloning of private repositories but also allows you to havepush access. This way you can type in your username and password to access your repositories.

Identifying yourself

Begin by identifying yourself to git (so all of your commits have this information) and logging in to GitHub:

git config --global user.name "Firstname Lastname"git config --global user.email "[email protected]"

133

http://git-scm.com

http://github.com/sunpy/sunpy

https://github.com/sunpy

https://github.com/signup/free

https://help.github.com/

https://help.github.com/

http://help.github.com/set-up-git-redirect

http://help.github.com/set-up-git-redirect


Forking SunPy

Each contributer to SunPy has their own copy of the SunPy master repo. When working on the code, changes are madeto this copied repo, and only when the changes are completed, and have been verified to work, are they pull requestedback to the upstream repo. GitHub provides a simple mechanism to setup your own personal repo by providing anoption to fork a repository. When you create a fork of a GitHub project, a copy of the repo will automatically becreated for you, and a link will be provided which you can use to download the code to your machine and beginworking on it.

To begin, fork the main SunPy repo on GitHub by clicking on the Fork button on the SunPy project page

Next, you need to download the forked repository. Clone the fork to your local machine, edit and run:

git clone [email protected]:your_username/sunpy.git

or:

git clone http://github.com/sunpy/sunpy.git

By default your fork of the repo on GitHub is identified by the name origin. In order to keep the fork up to date withthe main repo, it is useful to add it as a remote in git:

git remote add upstream https://github.com/sunpy/sunpy.git

To stay up to date you can grab the latest changes to the SunPy master using the commands:

git pull upstream master

This will merge the upstream code automatically with your code so you don’t need to worry about it overwriting yourchanges. After running either of these commands, your local copy of your personal repo is just a copy of the mainrepo. This is the same procedure that you will use in the future to keep yourself syncronized with the main repo. Tomake sure everything is setup correctly, let’s make some changes to our personal local repo and push those to ourpersonal repo on GitHub. Go ahead and modify one of the files, or create a new file (and then run git add).

Commit and push the changes to GitHub:

git commit -a -m "My first commit"git push

You local repo is now synced with GitHub and ahead of the main repo as it contains your personal contribution.Remember to commit after you’ve done a unit of work (i.e. often). This will make it easier for you (in the future)and everyone else to understand what you are doing. Also make sure to make your commit statements clear andunderstandable.

Installing SunPy

In order to use the version of SunPy located in your personal repository. You need to install it using the setup.pyscript located in the top-level folder. The setup.py script has several flags: :: develop : Installs SunPy and builds allexternal libraries. build or build_ext: (Re)Builds the external libraries. clean --all: Cleans all build files

Use the setup.py script like so:

sudo python setup.py develop

Conclusion

That’s it! You now have your own personal SunPy repo to develop on. You could hack away at it to your heart’scontent, pushing changes to your fork on GitHub to share with others and to ensure that you have a backup online.

But what about when you want to start contributing back to the main SunPy repo? That is the topic of the next section.

134 Chapter 20. Version Control

http://help.github.com/fork-a-repo/

https://github.com/sunpy/sunpy


20.2 Branches

Developers should create topic branches within their repos for most of their main coding. Every repo starts with asingle branch called master, which seldom needs to be used. Instead, work on any particular feature, bug, or portionof the code is done in its own separate branch. This way changes on any particular issue are isolated from otherunrelated changes. Users can even work on several different branches simultaneously.

To create a new branch run:

git branch branchname

To switch to the new branch:

git checkout branchname

(or alternatively, git checkout -b branchname will accomplish the above).

Developers should create new branches for the features they are working on. When they have finished making changesand the code has been tested and verified to be working well, the code can be merged back into the SunPy repo. Thisis usually done through something called a pull request.

20.3 Example Workflow

Before we get started

Here is an example workflow for a SunPy developer on any given day. Before beginning this tutorial, follow the aboveinstructions to grab a copy of the SunPy repo.

Grabbing other people’s changes

The first thing you want to do before you start coding anything new is to pull in the latest code that others have writtensince you last did any coding. To do this, run git pull:

git pull upstream master

This will ensure that you don’t edit a file that has changed since your last pull which will lead to merge conflicts lateron.

Code away

Assuming there are no merge conflicts (which shouldn’t happen unless two people are working on the same part of thesame file), then you are ready to begin coding. If there are conflicts check out our conflicts section.

Push your changes to GitHub

As you code away on your local repo, you will need to keep git aware of what you are doing and also your remotecopy up to date.

To add a file, create the file then run:

git add <yourfilename>

If you delete a file run:

git rm <yourfilename>

To move a file:

git mv <source> <destination>

20.2. Branches 135


To check to see if git is happy run:

git status

which will give you a report of what has happened so far. Once you are at a good stopping point you should “commit”your changes. This will provide you an opportunity to describe what you have done so far. To do this type:

git commit -a -m "description of your changes"

After doing this you are ready to push your changes to your repo online with the command:

git push

The local and remote copies of your repo are now synced.

Contributing to the main repo

Once you have made your desired changes, and committed and pushed your personal branch, you need to decidewhether or not to merge those changes back into the main SunPy repo. If the changes you made are finished andhave been tested and proven stable (see the testing section below), then they can be merged into SunPy. For now, letsassume that your changes are complete and they are ready to be added to the main SunPy repo. All contributed codeto SunPy must be submitted as a “pull request”. To do this go to the github website and to your repo (remember toselect the branch) then click on the “Pull Request” button (in the upper right hand corner next to the Fork button whichyou’ve used before). All initial pull requests must be done to the staging branch of sunpy (NEVER TO THE MAINBRANCH!). The staging branch is used as a test bed for new code. This will submit your code to a review. You willlikely receive some constructive comments on your code. To address these you can simply work on your code andpush those changes to your local repo. Those changes will be reflected in your pull request. Once a member of theSunPy dev team approves your pull request then your code will be merged into the main SunPy repo and your codewill be part of the main SunPy code. Congratulations!

And that’s it! It may seem like a lot at first but once you go through the motions a few times it becomes very quick.

Conflict resolution

It may so happen that when you try to sync with the main repo there is a conflict error. This means that someone elsehas been working on the same section of code that you have. In such cases, the merge command will issue a conflictwarning and will then expect you do the merge yourself. You can type:

git mergetool

to go through the conflicts. This command will likely open some merging tools which are already available on yourcomputer. For example, on Mac OS X, it will open FileMerge (if you have XCode installed). You can check on yourprogress by typing:

git status

Once you are done, you should then commit your changes, in this case the resolution of the conflict with:

git commit -m "Resolved conflict between my and online version of file.py"

You can then proceed to push this change up to your branch.

136 Chapter 20. Version Control

CHAPTER 21

Coding Standards

All code that is part of the SunPy project should follow The Style Guide for Python (PEP 8). Additionally, all codethat goes in the trunk should be checked using PyLint. PyLint is an open source tool which analyzes Python code andchecks for compliance with PEP8, as well as common coding errors and other potentially confusing or erroneous codestatements. Checking the SunPy trunk code this helps to ensure some baseline level of quality and consistency for thecode, and also helps to prevent potential problems from slipping through the cracks into the production code.

If you followed the installation instructions for devs, pylint should already be installed on your system. To run PyLinton a file, simply call pylint from the command-line, passing in the name of the file you wish to check:

pylint file.py

By default PyLint will print lines with potential problems along with a summary report. To disable the summary reportyou can add either -rn or --reports=no to the command:

pylint -rn file.py

Further, a paver task has been created so that all of the SunPy code can be checked at once:

paver pylint

The output from PyLint will look something like:

C: 87: Line too long (635/80)C:135: Line too long (98/80)R: 22:plot_fits: Too many local variables (22/15)R: 80:aia_color_table: Too many statements (59/50)W: 14: Unused import cmW: 16: Unused import Circle

Each line includes a line number, the category of the warning message, and a short description of the issue encountered.

The categories include:

• [R]efactor for a “good practice” metric violation

• [C]onvention for coding standard violation

• [W]arning for stylistic problems, or minor programming issues

• [E]rror for important programming issues (i.e. most probably bug)

• [F]atal for errors which prevented further processing

137

http://www.python.org/dev/peps/pep-0008/

http://www.logilab.org/card/pylint_manual


PyLint checks a wide range of different things so the first time you run PyLint on a file you will likely get a largenumber of warnings. In some cases the warnings will help you to spot coding mistakes or areas that could be improvedwith refactoring. In other cases, however, the warning message may not apply and what you have there is exactly asit should be. In these cases it is possible to silence PyLint for that line. PyLint warning messages can be disabled atthree different levels: globally (using a .pylintrc file), file-wide, and for a single line.

(To be finished...)

138 Chapter 21. Coding Standards

CHAPTER 22

Documentation

All code must be documented. Undocumented code will not be accepted into SunPy. Documentation should followthe guidelines in PEP 8 and PEP 257 (Docstring conventions). Documentation for modules, classes, and functionsshould follow the NumPy/SciPy documentation style guide. We provide an example of good documentation below oryou can just browse some of SunPy code itself for examples. All of the SunPy documentation (like this page!) is builtby Sphinx and must therefore adhere to Sphinx guidelines.

22.1 Sphinx

Overview

Sphinx is tool for generating high-quality documentation in various formats (HTML, pdf, etc) and is especially well-suited for documenting Python projects. Sphinx works by parsing files written using a a Mediawiki-like syntax calledreStructuredText. In addition to parsing static files of reStructuredText, Sphinx can also be told to parse code com-ments. In fact, in addition to what you are reading right now, the Python documenation was also created using Sphinx.

Usage

All of the SunPy documentation is contained in the doc/source folder and code comments. To generate the docu-mentation you must have Sphinx (as well as Numpydoc) installed on your computer. Enter the doc/source folder andrun:

make html

This will generate HTML documentation for SunPy.

Additionally, there is a paver command that can be used to accomplish the same thing:

paver prepare_docs

Which is a shortcut for the sphinx command:

sphinx-build source/ html/

For more information on how to use Sphinx, consult the Sphinx documentation.

The rest of this section will describe how to document the SunPy code in order to guarantee that well-formatteddocumentation will be created.

doctest

139



https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt

http://sphinx.pocoo.org/

http://docutils.sourceforge.net/docs/user/rst/quickstart.html

http://docutils.sourceforge.net/rst.html

http://www.python.org/doc/

http://paver.github.com/paver/

http://sphinx.pocoo.org/contents.html


The example codes in the Guide section of the docs are configured with the Sphinx doctest extension. This will testthe example code to make sure it runs correctly, it can be executed using:

sphinx-build -t doctest -b doctest ./ _build

from inside the doc/source folder.

22.2 Examples

22.2.1 Modules

Each module or package should begin with a docstring describing its overall purpose and functioning. Below thatmeta-tags containing author, license, email and credits information may also be listed.

Example:

"""This is an example module comment.

An explanation of the purpose of the module would go here and will appearin the generated documentation"""## TODO# Developer notes and todo items can be listed here and will not be# included in the documentation.#__authors__ = ["Keith Hughitt", "Steven Christe", "Jack Ireland", "Alex Young"]__email__ = "[email protected]"__license__ = "xxx"

For details about what sections can be included, see the section on documenting modules in the NumPy/SciPy styleguide.

22.2.2 Classes

Class docstrings should include a clear and concise docstring explaining the overall purpose of the class, required andoptional input parameters, and the return value. Additionally, notes, references and examples are encouraged.

Example (sunpy.map.Map)

"""Map(data, header)

A spatially-aware data array based on the SolarSoft Map object

Parameters----------data : numpy.ndarray, list

A 2d list or ndarray containing the map dataheader : dict

A dictionary of the original image header tags

Attributes----------header : dict

A dictionary representation of the image header

140 Chapter 22. Documentation

http://sphinx-doc.org/ext/doctest.html



date : datetimeImage observation time

det : strDetector name

inst : strInstrument name

meas : str, intMeasurement name. For AIA this is the wavelength of image

obs : strObservatory name

r_sun : floatRadius of the sun

name : strNickname for the image type (e.g. "AIA 171")

center : dictX and Y coordinate for the center of the sun in arcseconds

scale: dictImage scale along the x and y axes in arcseconds/pixel

Examples-------->>> aia = sunpy.Map(sunpy.AIA_171_IMAGE)>>> aia.TMap([[ 0.3125, 1. , -1.1875, ..., -0.625 , 0.5625, 0.5 ],[-0.0625, 0.1875, 0.375 , ..., 0.0625, 0.0625, -0.125 ],[-0.125 , -0.8125, -0.5 , ..., -0.3125, 0.5625, 0.4375],...,[ 0.625 , 0.625 , -0.125 , ..., 0.125 , -0.0625, 0.6875],[-0.625 , -0.625 , -0.625 , ..., 0.125 , -0.0625, 0.6875],[ 0. , 0. , -1.1875, ..., 0.125 , 0. , 0.6875]])>>> aia.header[’cunit1’]’arcsec’>>> aia.show()>>> import matplotlib.cm as cm>>> import matplotlib.colors as colors>>> aia.peek(cmap=cm.hot, norm=colors.Normalize(1, 2048))

See Also--------numpy.ndarray Parent class for the Map object

References----------| http://docs.scipy.org/doc/numpy/reference/arrays.classes.html| http://docs.scipy.org/doc/numpy/user/basics.subclassing.html| http://www.scipy.org/Subclasses

"""

22.2.3 Functions

Functions should include a clear and concise docstring explaining the overall purpose of the function, required andoptional input parameters, and the return value. Additionally, notes, references and examples are encouraged.

Example (numpy.matlib.ones):

22.2. Examples 141

https://github.com/numpy/numpy/blob/master/numpy/matlib.py


def ones(shape, dtype=None, order=’C’):"""Matrix of ones.

Return a matrix of given shape and type, filled with ones.

Parameters----------shape : {sequence of ints, int}

Shape of the matrixdtype : data-type, optional

The desired data-type for the matrix, default is np.float64.order : {’C’, ’F’}, optional

Whether to store matrix in C- or Fortran-contiguous order,default is ’C’.

Returns-------out : matrix

Matrix of ones of given shape, dtype, and order.

See Also--------ones : Array of ones.matlib.zeros : Zero matrix.

Notes-----If ‘shape‘ has length one i.e. ‘‘(N,)‘‘, or is a scalar ‘‘N‘‘,‘out‘ becomes a single row matrix of shape ‘‘(1,N)‘‘.

Examples-------->>> np.matlib.ones((2,3))matrix([[ 1., 1., 1.],

[ 1., 1., 1.]])

>>> np.matlib.ones(2)matrix([[ 1., 1.]])

"""a = ndarray.__new__(matrix, shape, dtype, order=order)a.fill(1)return a

For details about what sections can be included, see the section on documenting functions in the NumPy/SciPy styleguide.

22.3 Trouble-shooting

Sphinx can be very particular about formatting, and the warnings and errors outputted aren’t always obvious.

Below are some commonly-encountered warning/error messages along with a human-readable translation:

WARNING: Duplicate explicit target name: “xxx”.

If you reference the same URL, etc more than once in the same document sphinx will complain. To avoid, use double-




underscores instead of single ones after the URL.

ERROR: Malformed table. Column span alignment problem at line offset n

Make sure there is a space before and after each colon in your class and function docs (e.g. attribute : type, insteadof attribute: type). Also, for some sections (e.g. Attributes) numpydoc seems to complain when a description spansmore than one line, particuarly if it is the first attribute listed.

WARNING: Block quote ends without a blank line; unexpected unindent.

Lists should be indented one level from their parents.

ERROR: Unkown target name: “xxx”

In addition to legitimate errors of this type, this error will also occur when variables have a trailing underscore, e.g.,xxx_.

WARNING: Explicit markup ends without a blank line; unexpected unindent.

This usually occurs when the text following a directive is wrapped to the next line without properly indenting a multi-line text block.

WARNING: toctree references unknown document ‘...’ / WARNING: toctree contains reference to nonexistingdocument

This pair of errors is due to the way numpydoc scrapes class members.

22.3. Trouble-shooting 143

CHAPTER 23

Testing

This is a brief tutorial on how to write and run SunPy unit tests. SunPy makes use of the great package pytest for allof its testing needs.

23.1 Writing a unit test

Consider a simple module stuff.py that contains the simple function shown below.:

def double(x):return 2 * x

We can write a test case for this function by defining a new function containing the test (or tests) we want to perform.Suppose we want to check that the correct behaviour occurs when we pass a value of 5 to double(). We would writethe test function like this:

def test_answer():assert double(5) == 10

There are two things to note here. Firstly, names of test cases should always begin with test_. This is because pytestsearches for test cases named this way. Secondly, we use assert to assert our expectation of what the result of the testshould be. In this example, the test returns true and so the test passes.

The example given above is one in which the function and test reside in the same module. In SunPy, functions andtests are separated and the latter can be found in the sunpy/tests directory, organised by module. The conventionis to have one test module per science module, with the names for the test modules being the same as those for thescience modules prefixed with test_. For example, the modules util.py and multimethod.py in sunpy/util havecorresponding test modules test_util.py and test_multimethod.py.

There are some tests for functions and methods in SunPy that require a working connection to the internet. pytest isconfigured in a way that it iterates over all tests that have been marked as online and checks if there is an establishedconnection to the internet. If there is none, the test is skipped, otherwise it is run. Marking tests is pretty straightforwardin pytest: use the decorator @pytest.mark.online to mark a test function as needing an internet connection.

23.2 Running unit tests

To find and run all the SunPy unit tests, simply run

145


py.test

from the root of the SunPy tree (i.e. the directory containing INSTALL.TXT, sunpy, doc, etc.). This will produce a lotof output and you’ll probably want to run only selected test modules at a time. This is done by specifying the moduleon the command line, e.g.:

py.test sunpy/tests/util/test_util.py

for the tests for sunpy.util.util.

To run only tests that been marked with a specific pytest mark using the deocrator @pytest.mark (the the sectionWriting a unit test), use the following command (where MARK is the name of the mark):

py.test -k MARK

To exclude (i.e. skip all tests with a certain mark, use the following code (where MARK is the name of the mark):

py.test -k-MARK

Note that pytest is configured to skip all tests with the mark online if there is no connection to the internet. This cannotbe circumvented, i.e. it cannot be forced to run a test with the mark online if there is no working internet connection(rename the mark to something else to call the test function anyway).

To get more information about skipped and xfailed tests (xfail means a test has passed although it has been marked [email protected]), you need to use the option -rs for skipped tests and -rx for xfailed tests, respectively. Or use-rxs for detailed information on both skipped and xfailed tests.

23.3 When to write unit tests

A rule of thumb for unit testing is to have at least one unit test per public function.

23.4 Continuous Intergration

SunPy makes use of the Travis CI service. This service builds a version of SunPy and runs all the tests. It alsointegrates with GitHub and will report the test results on any Pull Request when they are submitted and when they areupdated.

The Travis CI server not only builds SunPy from source, but currently it builds all of SunPy’s dependancies fromsource as well using pip, all of this behaviour is specified in the .travis.yml file in the root of the SunPy repo.

23.4.1 New Functionality

For SunPy, we would encourage all developers to thoroughly cover their code by writing unit tests for each newfunction created.

Developers who want to take an aggresive approach to reducing bugs may even wish to consider adopting a practicesuch as Test Drive Development (TDD) whereby unit tests are written before any actual code is written. The testsbegin by failing, and then as they code is developed the user re-runs the tests until all of them are passing.

146 Chapter 23. Testing

https://travis-ci.org/sunpy/sunpy

http://en.wikipedia.org/wiki/Code_coverage


23.4.2 Bugs discovered

In addition to writing unit tests new functionality, it is also a good practice to write a unit test each time a bug is found,and submit the unit test along with the fix for the problem. This way we can ensure that the bug does not re-emerge ata later time.

23.4. Continuous Intergration 147


148 Chapter 23. Testing

Part VII

SSWIDL/SunPy Cheat Sheet

149


SolarSoft (SSWIDL) is a popular IDL software library for solar data analysis, and in fact, many parts of SunPy areinspired by data structures and functions in SSWIDL. Though IDL and Python are very different it sometimes helpsto consider how to translate simple tasks between the two languages. The primary packages which provide much ofthe functionality for scientific data analysis in Python are NumPy and SciPy. In the following we assume that thosepackages are available to you and that you are imported Numpy is imported as np with the following import statement:

import numpy as npimport scipy as sp

In the following examples, a and b could be arrays. For python the arrays must be numpy arrays which can be createdsimply through:

np.array(a)

where a is a python list of numbers.

Relational Operators

IDL Pythona EQ b a == ba LT b a < ba GT b a > ba GE b a >= ba LE b a <= ba NE b a != b

Logical Operators

IDL Pythona and b a and ba or b a or b

Math FunctionsIDL Python

cos(a) np.cos(a)log(a) np.log(a)exp(a) np.exp(a)

Math ConstantsIDL Python

!pi np.piexp(1) np.e

Arrays Sequences

IDL Pythonindgen(10)+1 np.arange(1,11, dtype=np.float)exp(1)

Array Creation

IDL Pythondblarr(3,5) np.zeros((3,5),np.float)intarr(3,5) np.zeros((3,5))dblarr(3,5)+1 np.ones((3,5))intarr(3,5)+9 np.ones((3,5))*9identity(3) np.identity(3)

Many more examples can be found on this page

151

http://sohowww.nascom.nasa.gov/solarsoft/

http://mathesaurus.sourceforge.net/idl-numpy.html


152

Python Module Index

ssunpy, 91sunpy.cm, 93sunpy.coords, 95sunpy.database, 97sunpy.database.attrs, 106sunpy.database.caching, 103sunpy.database.commands, 105sunpy.image, 107sunpy.image.rescale, 107sunpy.map, 109

153


154 Python Module Index

Python Module Index

ssunpy, 91sunpy.cm, 93sunpy.coords, 95sunpy.database, 97sunpy.database.attrs, 106sunpy.database.caching, 103sunpy.database.commands, 105sunpy.image, 107sunpy.image.rescale, 107sunpy.map, 109

155