monk documentation - read the docsmonk documentation, release 0.2a1 monk still is a project in early...

MONK DocumentationRelease 0.2a1

DResearch Fahrzeugelektronik GmbH

December 05, 2013

Contents

I Vision 3

II What Using MONK Might Look Like Soon 7

III Further Reading 13

1 Getting Started 151.1 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151.2 Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151.3 Patching MONK Locally . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2 Contributing 192.1 Tutorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.2 Reference Material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3 Coding Style Guide 313.1 In General . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.2 Whitespace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.3 Naming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.4 Files - Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.5 Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343.6 Exception Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.7 Imports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.8 The End . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4 Glossary 39

i

MONK Documentation, Release 0.2a1

If you have ever developed software on one architecture for another, then you have alreadyheard the term cross-compiling, meaning you compile software on one architecture for useon another. Using cross-compiling is quite helpful for developers who don’t use high levellanguages like Java for their codebase. In some situations it is the only way. How else wouldyou compile the first compiler for a new architecture?

MONK does the same for testing embedded systems. One machine (the test system) executestests by interacting with one or many other machines (target systems). The deployment of yourcode and configuration of the test environment can be done with the help of MONK.

What a cross-compiler is for your production code, MONK is for your test scripts, and insome regards even more!

Figure 1: Cross-testing with MONK

Contents 1


2 Contents

Part I

Vision

3


Here is how MONK was envisioned by its developers:

Figure 2: The vision of testing with MONK

The first approach shows the way most people consider writing software tests after first learningabout the concept. With more experience and some theoretic background they will probablyuse an approach as shown in column 2. This works fine in traditional software development.Typically, the test framework is harder to maintain than the test cases themselves, but it still ismanageable.

If, however, you add the complexity of embedded systems into the mix, then the tests becomeso complex that they themselves might not be maintainable anymore. Additionally, the amountof test code might become so big, that developers might be tempted to forget about automatedtests at all.

MONK was created to make impossible software tests for embedded systems possible, andeven enjoyable, when they would be frustrating otherwise.

5


6

Part II

What Using MONK Might Look LikeSoon

7


MONK still is a project in early development, and you will definitely find it harder to use thanin the following example. Still, it is always important to have a clear vision of what to expectfrom the future.

1 import monk_tf as monk2 import your_testframework as tf3

4 class MonkTestcase(monk.TestCase):5 """an example TestCase """6

7 def test_simple(self):8 self.prepare(9 env = monk.SingleUsecase(), #default

10 bins = (monk.Binary("https://github.com/DFE/example"),),11 logs = (12 monk.Log(13 path = "/var/log/messages",14 contains = ("regex1","regex2"),15 line_handling = monk.Log.LAST,16 ),monk.Log("~/.example.log",("regex3",))17 ),18 cmds = (monk.CMD(19 cmd_line = "example --args",20 contains = ("regex4",)21 ),)22 ) # end of prepare23 tf.change_whatever_you_want()24 self.execute()25 self.assert_env()26 self.clean()

The parameters of monk.TestCase.prepare() are the most powerful and most importantinteractions with the MONK framework. With these the developer of a test case tells MONK,what kind of test he wants to use (the Usecase) and then modifies it to his liking.

1 self.prepare(2 env = monk.SingleUsecase(), #default3 bins = (monk.Binary("https://github.com/DFE/example"),),4 logs = (5 monk.Log(6 path = "/var/log/messages",7 contains = ("regex1","regex2"),8 line_handling = monk.Log.LAST,9 ),monk.Log("~/.example.log",("regex3",))

10 ),11 cmds = (monk.CMD(12 cmd_line = "example --args",13 contains = ("regex4",)14 ),)15 ) # end of prepare16 tf.change_whatever_you_want()17 self.execute()

9


18 self.assert_env()19 self.clean()

The #default comment shows, that a line contains the default value. This means you don’tactually need to write that.


10 ),11 cmds = (monk.CMD(12 cmd_line = "example --args",13 contains = ("regex4",)14 ),)15 ) # end of prepare16 tf.change_whatever_you_want()17 self.execute()18 self.assert_env()19 self.clean()

The bins parameter contains the binaries you want to build. In this case a simple binarywill be downloaded from a github repository and then installed with $ make and $ makeinstall. Of course all of this might be modified according to your needs. You could supplyanother branch, commit, a binary instead of a repository or another build tool like Ant ordistutils. Also keep in mind that it is possible to simply add more binaries by adding moreobjects to the tuple.


10 ),11 cmds = (monk.CMD(12 cmd_line = "example --args",13 contains = ("regex4",)14 ),)15 ) # end of prepare16 tf.change_whatever_you_want()17 self.execute()18 self.assert_env()

10


19 self.clean()

The logs parameter contains the log files you want to watch on the target device. In theexample, there are 2 log objects because 2 different logs are to be watched. You have toprovide 3 attributes for each log: the path to the log in the file system, regular expressionsMONK should look for and a line_handling telling MONK how to treat lines createdbefore the start of the test. At the moment we intend to implement two possible states forline_handling: monk.Log.ALL, meaning all lines will be evaluated (which is the de-fault), and monk.Log.LAST, which ignores all existing lines when checking for your regularexpressions.


10 ),11 cmds = (monk.CMD(12 cmd_line = "example --args",13 contains = ("regex4",)14 ),)15 ) # end of prepare16 tf.change_whatever_you_want()17 self.execute()18 self.assert_env()19 self.clean()

The last important parameter of prepare() contains the commands that should be executedin your test. Each monk.CMD object consists of a cmd_line that is executed on the targetsystem by MONK, and regular expressions that the output should be checked against. As withmonk.Log objects there is the option to overwrite set line_handling, but at the momentthere are no menaingful usecases for this.

11


12

Part III

Further Reading

13

CHAPTER 1

Getting Started

1.1 Installation

To install MONK you need pip, a tool for installing and managing Python packages, whichyou can get via your system’s package manager, e.g.:

$ sudo apt-get install python-pip

Afterwards (or if it was present to begin with), you can use it to install MONK:

$ pip install monk_tf

This step might require sudo rights. You might also consider setting up MONK in avirtualenv. You can check that the installation worked the following way:

$ pythonPython 2.7.3 (default, Aug 1 2012, 05:14:39)[GCC 4.6.3] on linux2Type "help", "copyright", "credits" or "license" for more information.>>> import monk_tf>>> monk_tf.device.Device

If you also want to run unit tests for MONK you might want to read the developer instructions(page 19).

1.2 Usage

At this point of MONK’s development usage mostly relies on your ability to interact with theAPI and the development process (page 19).

When writing software tests, it is strongly recommended to structure them in two parts:

1. the Test Framework: which contains the code necessary to execute your tests.

15


2. the Test Cases: the actual test code.

This structure stays the same, when using MONK. MONK is not a replacement for the testframework, it’s merely an addition that simplifies both parts.

When writing tests with MONK, you will usually want to usethe monk_tf.devicetestcase.DeviceTestcase instead of theunittest.TestCase. An empty testcase might look like this:

1 #!/usr/bin/env python2 # -*- coding: utf-8 -*-3 #4 # Copyright (C) 2013 5 #6 # 7 #8

9 import monk_tf as monk10 import your_testcase as tf11

12 class TemplateTestCase(monk.devicetestcase.DeviceTestCase):13 """ an explanation of what this Testcase should test """14

15 def test_empty(self):16 #prepare17 #18 #execute19 #20 #assert21 self.assertTrue(True)22 #clean up23 #

Using the DeviceTestcase automatically enables some of the features currently imple-mented in MONK. For example, your target device will be rebooted and updated. Have a lookat the source code of devicetestcase if you are interested in more details.

1.3 Patching MONK Locally

As all users of MONK will have to be experienced software developers for the foreseeablefuture, there might arise a lot of situations, where you might want to apply local changes toMONK that the developers of MONK might not want to add to the official repository or whichwill simply not be added fast enough. Since Python itself works directly on its source code,you have many options to patch your code locally. However, keep in mind that there is onerecommended way to do this and although you are, of course, free to use other options as well,you will not get any support for them.

16 Chapter 1. Getting Started

http://docs.python.org/library/unittest.html#unittest.TestCase


1.3.1 1. Import and Extend (Recommended)

The best way to change existing modules is extending their classes and using your individualclasses instead. As an example, let us assume you want to make the conn attribute in Deviceoptional, in order to be able to add an individual connection later on, if needed. To do this,you should add another class (say DeviceOptionalConn) to the directory of your testframework, which inherits from monk_tf.device.Device:

import monk_tf as monk

class DeviceOptionalConn(monk.device.Device):pass

You may then go ahead to apply your changes as desired:

import monk_tf as monk

class DeviceOptionalConn(monk.device.Device):

def __init__(self, devtype):try:

self._setp = Device.Device_Types[devtype.lower()]except KeyError:

raise Exception("Unknown device type {}.".format(devtype))

self.bcc = bcc.Bcc()rst = self.bcc.reset if self._setup[’reset_cb’] == True else Noneatexit.register(self.__shutdown)

if self._setup["network_setup"] is not None:self.conn = connection.Connection(

network_setup = self._setup[’network_setup’],login = self._setup[’login’],serial_skip_pw = self._setup[’serial_skip_pw’],reset_cb = rst

)self._logger = logging.getLogger(__name__)

Instead of using monk_tf.device.Device you may then usetestframework.DeviceOptionalConn in your tests.

1.3.2 2. Patch the Install Folder

Another way would be to navigate to the directory containing MONK and directly overwritethe code. As MONK is an open source distribution, this should be straightforward. It is,however, impossible to foresee what will happen to your changes once you use pip,e.g. toupdate MONK. This method is thus not recommended.

1.3. Patching MONK Locally 17


1.3.3 3. Follow the MONK Developer Process

This is what you need to do if you want to have your changes applied to the MONK projectglobally. See Contributing (page 19) for further details about this point.

18 Chapter 1. Getting Started

CHAPTER 2

Contributing

Our project lives from your contributions. That’s why we took a lot of effort to make contribut-ing to MONK as easy as possible. Following the guide on this page leads you to your first stepsdeveloping with and for MONK.

2.1 Tutorial

2.1.1 Prerequisites

MONK is written in Python 2.7, which means it is mostly self containing and able to work onany operating system. Not all OS dependencies can be obvious, though. This means, that insome places you might have to investigate a solution by yourself. Feel free to drop us a line,though. Maybe we already faced your problem before or might include your solution as yourfirst contribution!

Most MONK developers work on Ubuntu, Suse and Fedora systems. If you have one of thosesystems, your chances are very high that you can follow this guide without any problems. Thisdocument itself is written on a Ubuntu 13.04 machine and all examples here are tested on thatplatform.

On most Linux distributions Python 2.7 should already be installed, so you have nothing to doin that regard. To test it, open a Terminal and look if that works without an error message:

$ pythonPython 2.7.4 (default, Apr 19 2013, 18:28:01)[GCC 4.7.3] on linux2Type "help", "copyright", "credits" or "license" for more information.>>> print "hi"hi>>> exit()

If it worked, than you already have Python and are ready to go. If not, you might find informa-tion on the Python website.

19

http://www.python.org/getit/


Another requirement is the version control system used for MONK. It’s called git and you canfind a lot of information about it in the git book. To follow this guide it’s not necessary to knoweverything about git, though. Don’t worry about it. The only thing you should have a look at isthe Installing Git chapter.

Then you will need 2 development packages. One is python-dev (might be called python-devel) and the other one is libssh-2-1-dev (sometimes called libssh2-devel). They are necessaryto build the Python package libssh2, which can be installed for you automatically, if you havethe 2 *-dev packages installed. Those are packages created by your operating system vendorand you should use his resources to get them. On Ubuntu 13.04 you can do the following:

$ sudo apt-get install python-dev libssh-2-1-dev

You will also need a Python tool, that helps with generalizing development environments. It’scalled virtualenv. If your operating system doesn’t come with a prebuilt virtualenv, you canfollow the virtualenv Installation Guide.

And of course you will need MONK’s source code. If you have installed git on your system,you can go to your preferred development folder and write the following line:

$ git clone https://github.com/DFE/MONK

This will create a new folder in that location and will download the source code inside.

Now you are ready to start developing for MONK. Great!

2.1.2 Starting The Environment

Now we want to discuss how to start our environment. It means an additional step before andafter your editing, but it comes with a lot of advantages. The tool that helps us here is virtualenvand what it does is creating a defined local environment. It resets standard variables like PATHand creates additional folders at the position of your virtualenv. Using it gives the followingadvantages:

• All developers using the same virtualenv initialization work in pretty much the sameenvironment.

• You separate your development environment from your system, which might be good, ifyou normally use another python version or another set of python packages.

• A lot of tools and services expect to run python code in a virtualenv. So it’s always goodto be prepared for it.

Working with virtualenv requires the following steps:

1. Create a virtualenv (only once)

2. Initialize the virtualenv (every time before you work)

3. Install the requirements (once in the beginning, and updated when changed)

4. Develop, test and do whatever you need to do.

5. Deactivate the virtualenv (every time after you are finished with your work)

20 Chapter 2. Contributing

http://git-scm.com/bookhttp://git-scm.com/book/en/Getting-Started-Installing-Githttp://www.virtualenv.org/en/latest/#installation


To create the virtualenv cd into your MONK folder, and then run the following command:

MONK$ virtualenv .

Now you run git status. You will see there are a lot of new folders which are not part ofthe MONK repository. Those were created by virtualenv and normally you don’t need to touchthem directly.

Every time before you start working on something you should run this command:

MONK$ . bin/activate(MONK)MONK$

If the activation was successful, then you should see the name of the folder inside braces beforeyour prompt. If you don’t see it, then please refer to the virtualenv documentation.

The next step you need to do is run the requirements. If you want to develop, you can install therequirements-tests.txt and if you want to develop you can install the requirements-develop.txt.Because it doesn’t hurt, we just install both now:

(MONK)MONK$ pip install -r requirements-test.txt(MONK)MONK$ pip install -r requirements-develop.txt

Remember that this step only needs to be done once in the beginning and when those file gotupdated. If there are no changes it doesn’t hurt to call them again, but it also shouldn’t changeanything. Now you can start working. In a later chapter we will look at things we can do here.

Now that you’ve tested that everything is okay, you want to install MONK inside of yourvirtualenv in a way, that makes it possible to test it and to develop it. This is achieved with thefollowing command:

(MONK)MONK$ python setup.py develop

After you are finished, you deactivate the virtualenv with this command:

(MONK)MONK$ deactivateMONK$

You see, that the name in brackets before the prompt is gone and you are back in the normalmode.

Now you know everything you need to start working with virtualenv. Awesome!

2.1.3 Running The Testsuite

Note: You need to install the requirements-test.txt file and execute python setup.pydevelop before you can go on with this chapter! See the previous chapter for a setup.

By now we have prepared our system, we retrieved the source code for MONK, and we sawhow to use virtualenv to start our development environment. As a first example of doing stuff,we want to run MONK’s unit test suite. It will also show us, if everything is installed correctly.

2.1. Tutorial 21

http://www.virtualenv.org/en/latest/


To run the test suite go into the virtualenv and then run the setup command for nosetests.nosetests is a tool that finds and runs testsuites while also collecting coverage data. This toolwill be automatically installed while running the setup command. You only need to worryabout it in case of errors or when you start to write unit tests yourself. In that case refer to thenosetests documentation.

The command:

MONK$ . bin/activate(MONK)MONK$ python setup.py nosetests

This command will install all required python packages locally into the folder. Read the outputcarefully. In general you should see successful attempts to download and install some packages,then a DEBUG output, were nose searches the path for all kinds of tests, then tests and theirsuccess, followed by the coverage data and a short summary that should be successful. If youexperience errors in this step it’s save to contact our mailing list ([email protected])!

To check if it is working, you can go into one file that is being tested and change something.Because we can be sure that there is a test for monk_tf/serial_conn.py, we will changethat:

(MONK)MONK$ vim monk_tf/serial_conn.py #or any editor of your choice

In that file add a line with raise Exception("Hello Error!") in the constructor (the__init__() method). If you run the tests again with python setup.py nosetestsfrom the project’s root folder (MONK in our example), the tests with SerialConn should allfail with that exception.

After you are finished, you can remove the Exception and don’t forget to deactivate your envi-ronment:

(MONK)MONK$ deactivate

The interesting point is, that you can now test on the go while developing. Not much differentfrom what an IDE would offer you. But in contrast to the IDE you also know that you have anequal environment to other developers, users and your CI server, in our case Travis CI.

2.1.4 Reading the Documentation

The MONK documentation is written in Sphinx. Sphinx is a set of tools, which compilesdocuments in the restructuredText markup language to different formats like HTML and PDF.reST is a language that is both written for compilers to parse, as well as for humans to read. Thatmakes reading .rst directly in your editor possible. Text editors like VIM will even highlightthe reST format for you. So the simplest way to read the documentation will be directly in thesource tree:

(MONK)MONK$ cd doc/source#read this page’s source file:(MONK)MONK$ vim contributing.rst


https://nose.readthedocs.org/en/latest/mailto:[email protected]:[email protected]://en.wikipedia.org/wiki/Continuous_integrationhttps://travis-ci.org/DFE/MONKhttp://sphinx-doc.org/


Another way to read the docs, is on the corresponding Github page, but make sure that you areat the right commit. The advantages of the website are that you don’t need any environmentfor reading it, Github already parses it to HTML for you and you can even edit the text there.For changing the documentation that might be a solid way to handle it, although we didn’t testit until now!

The more advanced way is to build the docs yourself to PDF or HTML. This will be explainedin the next section. The advantages here are that you read all the warnings and error messagesand have full control over every part of your documentation. If you want to be a successfulMONK developer this is definitely the way to go. It is not the easiest, though. But we trust youcan handle it!

2.1.5 Building the Documentation

As always the first step is to go into your MONK repository and activate the virtualenv:

$ cd MONK$ . bin/activate(MONK)MONK$

Then you go into the doc folder:

(MONK)MONK$ cd doc(MONK)doc$

When you look into that folder you see that there is a Makefile and that contains tasks foreverything you could want to do. In general you just want the html docs. In that case you call:

(MONK)doc$ make html

If you want to build everything and decide later what to use, then:

(MONK)doc$ make

Keep in mind that building everything requires some latex related system libraries. Until nowwe have no comprehensive list about those and fall back to simply install the Lyx Latex editor,which automatically installs all required packages (and some more):

(MONK)doc$ deactivatedoc$ sudo apt-get install lyxdoc$ . ../bin/activate(MONK)doc$ make

We seldomly use PDF by now, that’s why nobody got to improve that part. So if you use thatheavily, then we would appreciate your commits there!

The output you see then begins by trying to delete old folders that might still be lying around.Then Sphinx is started and the output you see is Sphinx going through it’s compilation process,followed by the output from latexpdf a tool to generate PDFs from text files.

If everything ran successfully, you should find the following documents, which can be openedwith a browser and a pdf reader respectively:

2.1. Tutorial 23

https://github.com/DFE/MONK/blob/dev/doc/source/contributing.rst


MONK/doc/build/html/index.htmlMONK/doc/build/latex/MONK.pdf

For problems with building the HTML you should refer to the Sphinx Documentation. Forproblems with building the PDF we can’t help you. Sorry! But we wish you a great deal ofluck! :)

In the end, don’t forget to deactivate your virtualenv:

(MONK)doc$ deactivatedoc$

2.1.6 Change Example: Creation Of This Tutorial

As a last point of this tutorial I want to discuss how the changes were made to put this tutorialhere in this place. This part is just for reading. I hope you understand that it’s not possibleto have any specific change in the project, that can be always go through the whole MONKdevelopment process each time someone wants to go through the tutorial. We just hope thatafter reading this you will have the confidence to go ahead and do something yourself withMONK. If you hit any road blocks on the way, don’t hesitate to ask for our help!

History Lesson

This tutorial was developed, because one of our developers felt, that the previous version ofthis document wasn’t very convincing to new users. It was basically a list of details about thedevelopment process, with too much information and too little guidance. So when the time wasright and enough changes to the development process were made, this tutorial was created fornew comers, to get an easier step into MONK and open source development. Although there isprobably still a lot to be done better, we hope we at least succeeded in convincing you to workwith us on MONK.

The Preparation

The problem was discovered by one coworker with the Github shorthand erikb85. When henoticed the problem, he started an Issue on our developer platform Github. In that he describedthe problem. Because in this case he already knew the solution to the problem, he desribed thatas well. In most cases the solution won’t be clear. Then that part of the Issue description staysempty until a solution is found.

After that he fetched the newest version of MONK from Github. Because he already has adevelopment environment, it wasn’t necessary to clone MONK:

#go to the MONK folder$ cd MONK#make sure to be on the right branchMONK$ git checkout dev#pull updates from GithubMONK$ git pull --rebase


http://sphinx-doc.org/


Then he created a feature branch for his new commits and added it to the Notes section in theIssue:

MONK$ git checkout -b f-docs-devenv

Now he started the development environment and went to the documentation folder:

MONK$ . bin/activate(MONK)MONK$ cd doc(MONK)doc$

Then he started to edit the contributing.rst and built the html documentation to check his results:

(MONK)doc$ vim source/contributing.rst(MONK)doc$ make html...(MONK)doc$ chromium-browser build/html/contributing.html# repeat until happy or time runs out

When he left work, he committed his changes in form of a WIP commit, always mentioning theGithub Issue with #36. This way Github always knew were the commit belongs to and couldlist it in the Issue:

(MONK)doc$ git add source/contributing.rst(MONK)doc$ git commit -s#commit message:WIP: contributing.rst: Rewrite for development env

See Github Issue #36.

Signed-off-by: Erik Bernoth #end of commit message#be careful with the following command, because it influences the internal#structure of your git repository as well as GitHub’s(MONK)doc$ git push -f origin f-docs-devenv

Now the most up-to-date version is always online and all commits show in the GitHub Issue.Some major decisions are also written as comments to the Issue. An example is the comment,which discusses changes to the topic issue, which seemed meaningful at some point whileworking on solving it. This is also a place for reviewers, interested users and other developersto discuss their ideas about the problem.

After this tutorial was finished, it got the label reviewable and other developers took a look intoit. A reviewer will checkout the same branch (not a commit specifically) and look at the results.If code was changed he will look what happened and if the code still works as expected, if unittests are there, if they pass and if the coding style guide was followed. If he is happy, he willwrite a comment with a simple ACK inside. Sometimes it gets more complicated and he willcommunicate with the developer directly. The result will be as in this case more commits by thereviewers or some changes by the developer himself. If 2 reviewers give an ACK in commentsor directly to the developer he or a maintainer will go through the commits and add a line foreach reviewer to the commit message, saying that this commit was checked by whom:

2.1. Tutorial 25


TODO: add Acked-by line according to Issues #36 reviewers. If you are areviewer and read this, add an Acked-by example with your name here,please.

Then a maintainer will close the issue and merge the feature branch with the dev branch. Nowthe Issue is solved.

As you might have noticed, the last steps were explained more broadly. They require a deeperunderstanding of git and at the point were you get there, a MONK developer will probablyalready help you, anyway.

2.1.7 Final Words

Okay, that closes the tutorial. Now you have learned all the steps in the development processand seen one example of how it was applied. You should know why and how to use virtualenv,git, Github and MONK. Some more detailed descriptions follow underneath and in the nextchapters. We hope, you could learn something and have a good time developing on MONK.

Thanks for reading to this point!

2.2 Reference Material

2.2.1 Communicating with the Developers

You can communicate with other MONK developers in one of the following ways:

1. the Mailing List: You can ask questions at [email protected]. You can alsohave a look at the archives to see old conversations.

2. the Bug Tracker: If you have encountered a specific problem you want to work on, youcan create an Issue here. It is also possible to work with Github Pull Requests.

3. the Stammtisch Meeting: If you happen to be in Berlin on any Tuesday afternoon at2:30 pm, you are welcome to join our regular meeting on all topics concerning MONKat:

DResearch Fahrzeugelektronik GmbHOtto-Schmirgal-Str. 3D-10319 BerlinGermany

2.2.2 The Development Process

If you want to see your changes applied to MONK, the best way is to first communicate yourideas with the other MONK developers. You should only start development, if they agree the


mailto:[email protected]://groups.google.com/a/dresearch-fe.de/forum/?fromgroups#!forum/project-monkhttps://github.com/DFE/MONK/issues


change makes sense. Otherwise, your work might turn out to be a waste of time.

If you have write access to the repository (which is only supposed to happen if you are a regularmember of the DFE group), you should simply fork, create an Issue and track your changes ina feature branch.

If you do not have write access, you will have to use pull requests. Remember that a featurebranch will automatically be generated for your pull request.

Step-by-step guide:

1. Start from the dev branch: You always need to ensure your commits can be appliedwithout problem on the current HEAD of the dev branch.

2. Create meaningful commits: Commits should adhere to the requirements described inCommit Policy (page 28). An important point is, that they should always contain onlyone meaningful change.

3. Create a pull request: Follow Pull Request Guide on GitHub.

4. Receive acknowledgements: Before a maintainer may accept your changes, you willneed to find two DFE developers to acknowledge them. These two people should bementioned within your commit message, as described later.

5. Get it merged: If you have a pull request or Issue containing your problem, your solu-tion, and the commits containing the two Acked-Bys, get one of the maintainers to mergeit.

If you need to change anything on your commits, e.g. to add some Acked-by lines, it is okayto rebase -i or commit --amend your changes on your feature branch or pull requestas nobody else is supposed to work on the same feature branch anyway.

2.2.3 Branching Model

There are four different kinds of branches in MONK:

1. the Master Branch is the stable branch. It should only contain project states that maybe distributed to users.

2. the Dev Branch contains all currently accepted features. Changes here might be moreunstable than in the Master Branch and might not be properly documented. Using thisbranch might sometimes break your code, but as a MONK developer you will alwaysneed to start implementing your features and bug fixes from the HEAD of this branch.

3. Feature Branches are used by members of the DFE group to work on bug fixes and newfeatures.

4. Pull Requests are used by external MONK developers who want to get their changesmerged into MONK.

2.2. Reference Material 27

https://github.com/DFEhttps://help.github.com/articles/using-pull-requestshttps://github.com/DFE


2.2.4 Acceptance Criteria for the Dev Branch

dev contains the latest changes that are considered to be working by their creators and at leasttwo reviewers. To continually ensure good quality of the code, some requirements must be metfor a commit to be allowed into dev:

• All commits refer to an Issue on GitHub.

• The Issue contains an understandable description of the problem that enables reproduc-tion, if necessesary.

• The Issue contains an understandable and assessable description of the solution.

• All code changes are well documented, formatted according to the coding standards andfollow high quality standards even in areas that are not specifically mentioned here.

• Code changes are unit tested with 100% statement coverage for the changes involved.

• If necessary and possible: integration tests and documentation adapted accordingly.

• Two DFE team members have acknowledged the solution’s successful and complete im-plementation.

These requirements can be overruled only by 100% acceptance of all developers, reviewers andboth maintainers for a single Issue, if considered necessary.

Changes to this list of rules can only be achieved by acceptance at the Stammtisch Meeting.

2.2.5 Acceptance Criteria for the Master Branch

The dev branch may be merged into the master branch whenever the Stammtisch decidesthat MONK’s current state warrants doing so.

2.2.6 Commit Policy

All commits are are expected to adhere to the following requirements:

• KISS principle: Commits should contain one meaningful change and that one changeonly. This does not mean you should only put changes in one file or one deletion/insertioninto one commit, but that all of the changes should belong together to do one meaningfulthing.

• All English: All relevant texts in MONK (like this documentation) are written in Englishso that non German speakers may use it. This, of course, applies to commits as well.

• Summary Upfront: The first line contains a short summary with no more than 50 char-acters. This line must be followed by an empty line.

• Descriptive Content: The following paragraphs contain the long description of the prob-lem as well as its solution.


https://github.com/DFE/MONK


• 72 characters per line: Comment messages should not exceed 72 characters per line,except for longer URLs, quotations or messages that would be unintelligible in someother way when split.

• Refer to an Issue on GitHub: If you have not done so already within the description,this would be a good place to specify which Issue on GitHub your commit belongs to.

• Signed-off-bys: After the long description all developers involved in creating this com-mit should on seperate lines beginning with Signed-off-by:. These lines shouldinclude their names and email addresses.

• Acked-bys: Afterwards all people who checked the quality of the commits should bementioned in the same fashion with Acked-by: lines.

Finally, a complete example doing everything right:

some_file: limit checks debugged

There was a problem with the limits in this line. They excluded 0,which is still an acceptable value. This led to unexpected errormessages in the GUI.

The bug was fixed by changing ‘>0‘ to ‘>=0‘.

For more details see GitHub Issue #312.

Signed-off-by: Peter Parker Acked-by: Bruce Wayne Acked-by: Clark Kent

2.2. Reference Material 29

CHAPTER 3

Coding Style Guide

This guide summarizes how we MONK developers think the source code should be formatted.Consider these guidelines advice rather than law. Sometimes you might hit a point, where thecurrent version of the document can’t help you. In this case you can do the following:

1. Check whether the newest version of this document contains any changes concerningyour situation. Also consider building the docs yourself.

2. Refer to PEP-008, the official style guide for Python core development. The Python Coreproject is, of course, a lot bigger than MONK, meaning you might find something there,that has not been addressed in MONK yet. Place a comment in the code explaining whyyou were not able to follow MONK’s style guide. If possible, create an Issue and try toget the style guide changed accordingly.

3. Discuss with other developers whether there might be a way that you are not aware of.

4. Do what makes most sense in your context. Use comments extensively and find mean-ingful ways to tell pylint and other tools that this line is to be treated as an exception.

3.1 In General

• Avoid features specific to your python binary. That includes your language version aswell as the interpreter you are using, i.e. CPython, Jython or Pypy.

• Every line should consist of 79 characters at most.

• No line should contain more than one statement.

• All classes should be New Style Classes.

• Do not use getters/setters. Use properties instead.

• When comparing singletons like None always use is instead of ==.

• When overwriting comparator functions like __eq__, ensure that you overwrite all ofthem.

• Every function consists of 30 statements and 12 branches at most.

31

http://www.python.org/dev/peps/pep-0008https://github.com/DFE/MONK/issues/newhttps://jamiecurle.co.uk/blog/79-characters-or-less/http://www.python.org/doc/newstyle/http://stackoverflow.com/questions/6618002/python-property-versus-getters-and-settershttp://stackoverflow.com/questions/2209755/python-operation-vs-is-not


• Use the with clause to open context managers, e.g. when you use open().

• Write your code as type independently as possible, since we should reap the benefits ofdynamic typing while using it. E.g. use “duck testing”.

• Use isinstance() instead of type() comparisons because it also checks for inher-itance.

• When checking for emptyness of lists, consider that empty lists are already False. Thereis no need to check for their length.

• Do not check for the True-ness of a value explicitely with if bool_val is True:because it is already True or False. Use if bool_val: instead.

• Callable Python scripts should always contain a main() method. This way your codewill not only be executable by your script but can also be used by other python moduleswhich import yours. If you do not include one, and instead put your code directly intothe if, then only your script can make use of that code, which is not preferred. Example:

def main():print("hello world")

if __name__ == ’__main__’:main()

3.2 Whitespace

In Python whitespace has meaning. Thus, treating whitespace should receive even more con-sideration than in other languages. Whitespace means:

• (a space character)

• \t (a tab character)

• \n (a line feed character; typical for most Unix-like operating systems)

• \r\n (a carriage return character, followed by a line feed character; typical for Windows)

Rules:

• 4 spaces indentation

• no tabulator characters, only spaces

• New lines are \n characters.

• Classes, functions, and blocks should be separated by 2 empty lines each.

• no whitespace in params, brackets, and braces

• no whitespace before commas, semicolons and colons

• no whitespace before ( in function and class headers

• no whitespace before list indexing, e.g. list[5], slice[3:4]

32 Chapter 3. Coding Style Guide

http://docs.python.org/library/functions.html#openhttp://c2.com/cgi/wiki?BenefitsOfDynamicTypinghttp://c2.com/cgi/wiki?BenefitsOfDynamicTypinghttp://en.wikipedia.org/wiki/Duck_typing#Concept_exampleshttp://docs.python.org/library/functions.html#isinstancehttp://docs.python.org/library/functions.html#typehttp://stackoverflow.com/a/1549854/131120http://stackoverflow.com/a/1549854/131120http://www.python.org/dev/peps/pep-0008/#programming-recommendations


• no whitespace in default param assignments, e.g. def function(a=13):

• exactly one space around assignments/operators

A complete example of correct whitespacing:

def function(a, b=5):my_list[b] = aprint(my_list[2:3])

def another_function(d, e):f = d + efunction(a=f, b=e)

3.3 Naming

Files, folders, modules, classes, functions, and attributes are to be named in the followingfashion:

• files, folders, modules, packages: all_lower_case

• variables (including global variables): all_lower_case

• constants (convention only, not enforced by interpreter): ALL_UPPER_CASE

• classes: BigCamelCase

• functions, methods: all_lower_case

• Naming something with a leading underscore declares it to be soft private (e.g._like_javas_protected).

• Naming something with two leading underscores makes it private, i.e. only usable insidethis class (e.g. __only_for_me).

• Use of double underscores at beginning and end should be avoided, since Python itselfuses this convention for special names (e.g. something.__doc__)

• Use self to reference the object calling a method.

• Use cls to reference the class in class functions.

3.4 Files - Modules

• Files are to be encoded using utf-8.

• Every file starts with the following 2 lines:

#!/usr/bin/env python# -*- encoding: utf-8 -*-

3.3. Naming 33


The first line calls the python binary the way you would from the shell. This way it ispossible to use the script in a virtual environment. The second line tells other tools aboutthe expected encoding of this file.

• Following these there is an empty line followed by the copyright and licensing text.

• After this general information there is the module docstring.

• The docstring is followed by the imports in this order:

– general imports from the Python standard library

– imports from frameworks

– imports from your own project

• Next are global variables and constants, if necessary (use of them is generally discour-aged).

• Next, there is the main part containing the class.

• If required, the main() function follows below the class.

• At the end there is the call of the main() function:

if __name__ == ’__main__’:main()

3.5 Comments

In general it is best to write code as self-explanatory as possible. Yet sometimes you can’t getaround writing comments, to make things clear. Here are some situations in which you shouldwrite a comment:

• Each module, class, and function needs to be accompanied by a docstring.

• Whenever you find yourself writing code that cannot be understood without explanation(although you might want to consider refactoring the code instead).

• If you want pylint or other static code checkers to ignore a piece of code that violates theexisting coding style.

• If you took a sizable piece of code from a book or a website, reference the source (andbe sure to check the license of that code).

• Reference dependencies between code parts, which might not be obvious. This is crucial,if someone wants to make changes on either side, as these might introduce new bugscaused by unknown dependencies.

Comments themselves must also be written uniformly. Therefore you should follow theserequirements when writing a comment:

• Each comment is written in English.

• Each comment describes something underneath, excepting docstrings which also partlydescribe code that is above them, e.g. in function doc strings.


http://www.python.org/dev/peps/pep-0257/#what-is-a-docstring


• To put it the other way around: comments are not appended to a line of code, insteadthey are written above the line they describe:

#good comment about do_somethingdo_something(1,2,3) #bad comment about do_something#bad comment about do_something

• Exceptions to this are docstrings which usually are directly underneath of what is to bedescribed:

class Something(object):"""This is a docstring.

It is put directly underneath the class definition."""

def __init__(self):"""This is also a docstring.

It is put directly underneath the method definition."""

#This is not a docstring, thus it is put above its target.do_something(4,5,6)

• Each comment is indented the same way as the text underneath, not as the text above it.This is because indented text is treated as inside a function, while the next line of thesame indention is treated as outside the definition, as with every other Python code:

def do_something(*args):"""This is the correct way to indent a docstring.

The Python compiler understands that this belongs to the functiondeclaration above."""

def do_else():"""This is the wrong way to indent a docstring.

It won’t even be recognised as a docstring."""

• Texts in comments are parsable by the Sphinx documentation generator. This is a com-plicated issue, so do not expect to find a 1 minute solution here! Seriously, start readingthe Sphinx website, if you need to write more than one or two lines of documentation!

• Single line comments begin with a hash (#) character.

• Multiline comments, including docstring, adhere to the following rules:

– They start with three straight double quote (""") characters.

– The first line of the comment starts directly afterwards and is treated as a short

3.5. Comments 35

http://sphinx-doc.org/http://www.python.org/dev/peps/pep-0257/#what-is-a-docstring


summary by many tools.

– Newlines are used meaningfully inside and there should be at least one emptyline between the summary (the first line) and the verbose documentation (therest of the docstring).

– The commenting text is not followed by an empty line.

– The end is marked by a line consisting exclusively of three straight doublequote (""") characters.

– They should consist of complete sentences.

– They should contain descriptions for elements like parameters, as described inthe Sphinx Docs.

• Comments for version control systems should be added to the __version__ magicvariable.

3.6 Exception Handling

In most cases exception handling should be done like in any other proper Python project. Hereare some things to consider:

• Use Exception objects and raise to initiate exception handling.

• State the Exception classes explicitly in except clauses because explicit is better thanimplicit.

• Minimize the number of lines in try clauses to avoid Pokemon exception handling.

• In Python, it is encouraged to use function parameters as expected instead of forcingcertain types. If the delivered objects do not have the expected capabilities and there isno sensible way to handle this, then raise exceptions:

def u_gonna_quack(duck):"""I’m expecting a duck but don’t force it to be one."""try:

duck.quack()except AttributeError:

print("The param duck can’t quack.")

3.7 Imports

As stated above, import calls should be written below the module docstring, above the con-stant/global variable definitions, and in the following order:

• general imports from the Python standard library

• imports from frameworks


http://sphinx-doc.org/domains.html#info-field-listshttp://sphinx-doc.org/domains.html#info-field-listshttp://www.codinghorror.com/blog/2012/07/new-programming-jargon.html


• imports from your own project

Different imports should be on different lines. An exception to this are statements like fromabc import x,y,z, making different imports from the same source. This is discouragedfor other reasons, though. Instead of from abc import x,y,z you should use importabc and then refer to abc.x later on. This way it is easier to identify where something comesfrom, even though it is a little more work writing. Always remember explicit is better thanimplicit.

3.8 The End

You’ve read it all. I’m so proud of you!

3.8. The End 37

http://www.python.org/dev/peps/pep-0020/http://www.python.org/dev/peps/pep-0020/

CHAPTER 4

Glossary

cross-compiling The act of compiling software on one architecture, i.e. a personal computer,for another architecture, i.e. an embedded system.

embedded system A computer highly adapted to a specific usecase, often used inside cars,trains, airplane and the likes.

reST reST is the short form of restructuredText, which is a markup format for text files. Muchlike Wiki markup or Markdown it is a language with 2 goals: easily readable and write-able source code, as well as a logical structure understandable by a compiler program.Writing reST source files is often not too different from simply writing a text into a nor-mal txt file. But with the logical structure together it can be compiled to formatted HTMLor PDF. Writing in those markup languages is also considered easier then writing HTMLdirectly. reST itself was created for documenting the Python core libraries. Today itdeveloped to the quasi standard for documenting all kinds of Python projects and someprojects in other languages use it as well. See the reST website for more details.

target vs development system Embedded systems are highly specialised for one task and thattask alone. Because these tasks are unrelated to software development most of the time,the software for these embedded systems needs to be developed on a separated system,which is often a normal personal computer. The system the software is developed on iscalled the development system and the system the software is developed for is called thetarget system.

test framework A well maintained test scripting project consists of two parts. One part arethe test scripts themselves and the other part is a test framework of helper classes andfunctions that should help in clarifying the test code and take care of some of the com-plexity.

test scripting writing tests in a programming language. Other ways of testing software couldbe record&replay or manual testing.

Usecase Contains all the default configurations of a general branch of activities with the frame-work. Every time you use MONK, you will also apply a Usecase, if you know about itor not. The most important and simplest Usecase is the monk_tf.SingleUsecase.

39

http://en.wikipedia.org/wiki/Help:Wiki_markuphttp://daringfireball.net/projects/markdown/http://docutils.sourceforge.net/rst.html

I VisionII What Using MONK Might Look Like SoonIII Further ReadingGetting StartedInstallationUsagePatching MONK Locally

ContributingTutorialReference Material

Coding Style GuideIn GeneralWhitespaceNamingFiles - ModulesCommentsException HandlingImportsThe End

Glossary

monk documentation - read the docsmonk documentation, release 0.2a1 monk still is a project in early...

Documents