SOFTWARE CRAFTMANSHIPWhat you should know before typing a single line of code

Copyright © 2016 Jeffrey Chongsathien

INTRODUCTIONThese pages comprise a collection of thoughts and ideas that I've developed from personal experience and accumulated knowledge in software engineering.

Much of it is about developing a culture, whether as an individual or for a group.

My hope is that by adopting the ideas and habits described here, someone starting out in the field becomes a better engineer, and produces better work product, more quickly than they otherwise would. I would say that this is more important than, say, learning the syntax of any one particular language or platform.

“Software Engineering is a craft more than a science. Software Engineers are craftsmen. We use a variety of tools. We have good and bad habits. We have technique. We become better as we experience more and make mistakes. We discern the gulf between theory and practice. We work well alone and with others. We blend form and function, the aesthetic and composition. We seek order but generate chaos through creativity. We traverse simplicity, complexity, consistency and diversity.”

Jeffrey Chongsathien

CORE PRINCIPLES☐ Break everything up into manageable pieces

☐ Don’t reinvent the wheel

☐ Ugly, unstructured, undocumented anything is expensive (“Technical Debt”)

☐ No-one owns anything. The illusion of ownership ⇒ silos ⇒ disaster

☐ Never ask anyone to do something you’re not prepared to do yourself

☐ Taste your cooking... before serving it to someone else

☐ Good Estimation = Experience + Pessimism

☐ The best argument wins

☐ Everyone gets their say. To paraphrase Steve Jobs, hire smart people to tell you what to do

☐ “I don’t know” is a valid answer

☐ “I disagree” is a valid response

☐ No egos; narcissists are not fun to work with

☐ No false economies; cheap is expensive

☐ Deal with issues early, or it’ll be like running with a stone in your shoe

WORKING GUIDELINES☐ Work sustainably; Burn-out helps no-one

☐ Keep a work diary

☐ Show up to meetings on time

☐ Weekly meetings are for open discussion of ideas, problems, solutions and to look ahead to the next 7 days

☐ No daily stand-ups - it’s a terrible substitute for team chemistry

☐ ‘Done’ includes testing and feedback

☐ Code that should compile, must compile

☐ If someone needs help, you help

DOCUMENTATION - BEST PRACTICEOver the course of my career, I've seen projects and companies brought to their knees by one thing - lack of documentation. In fact, I'm convinced that poor or absent documentation is the leading cause of total development costs spiralling out of control (if not complete project failure).

Why Document?

☐ Institutional memory

☐ Avoid silo-ing of information

☐ Avoid dependency ”What if X is hit by a bus?!"

☐ Prevents multiplying cost of development

Documentation is…

☐ A trail of breadcrumbs for the next person

☐ A template to success for you

Documentation is NOT the aim

☐ Too much and it chokes productivity

☐ Too little and the project falls into the abyss of abandon-ware

☐ Documentation should be minimal but sufficient

Key Components

☐ Several key ‘living’ documents that are continually referenced and updated

☐ Letterhead, presentation slides and other templates

☐ README files

☐ Architectural Diagram

Style

☐ Have a checklist mentality

☐ Emphasise diagrams

☐ Use plain, simple language

☐ Brevity is golden

Good Habits

☐ Make notes as you go along; if you leave it to the end, you’ll fail

☐ Record how to reproduce any process

☐ Post-It Notes are your colour-coded, modular friends that make it easy to replace and rearrange

Source Code

☐ Use a documentation generation tool

☐ As a general rule, write an overview for each class/module, not individual functions or variables

☐ Inline comments save lives

LEADERSHIPIn many software engineering contexts and circumstances, one needs to exhibit good leadership qualities, whether as the appointed leader, just filling in, or picking up the slack on some piece of work. Here are some of my thoughts on leadership.

☐ Have a vision of where to go

☐ Have a strategy of how to get there

☐ Communicate both clearly and simply

☐ ‘Push’ information

☐ Never ask anyone to do anything you’re not prepared to do yourself

☐ Address issues pro-actively and aggressively

☐ Run for your life from the General that doesn’t dine with the troops - he is not aware and he does not care!

☐ The non-coder “software manager” is toxic

☐ Coach, guide - don’t manage

☐ Set the standard by example

☐ Be engaged and responsive

☐ Be cool(-ish) under pressure

WALL OF INFAMYI've had the good fortune of working with some amazing, talented people who were a joy to be with, which is important when you spend as much time with them as you do your family (if not more). Unfortunately, I've also had the misery of working with some real nut-cases, which is common in the tech industry. Here are some traits to look out for:

☐ Sociopath

☐ Narcissist

☐ (Aggressively) Insecure

☐ Evasive

☐ Whiny, Needy

☐ Incompetent

TEAMS

Location

☐ Create an aesthetically pleasant space (natural light and materials) from a reference design

☐ Get lots of large whiteboards and use them frequently

Communication

☐ Face-to-Face > Messaging > Voice > Email

Self-Organising

☐ Put a unified BITS (Bugs, Issues, Tasks, Stories) management system in place. Your team will self-organise around it.

Empowerment

☐ Encourage a results-oriented culture of independent action, punctuated by meetings and demos.

Cross-Functional

☐ Get a variety of skills, experience and personalities.

☐ Prefer adaptable people.

Getting great people can be the difference between success and failure. A great hire produces an outsized return on investment. Since people are the most expensive part of any business, a bad hire is an expensive mistake. It’s worth making the effort to get it right.

However, outside of the world of politics, one of the greatest repositories of bad ideology is the interviewing process for software engineers/programmers. Technical exams, puzzle quizzes, coding tests... so much wasted time, effort and misery for so little efficacy.

INTERVIEWING

1st Filter: References

☐ Get the references first and dig deep. How credible are they? Is the narrative consistent? What’s the general trend?

☐ Get code samples (unfortunately, rarely possible due to IP issues)

☐ Study their web and professional social media presence.

2nd Filter: In-Person Interview

☐ Have more than one person in the room

☐ Ask him/her to brainstorm the qualities that make a great software engineer - habits, ideas, principles, personality.

☐ Systematically discuss past projects. For each project, talk personal involvement, successes, failures, knowledge, around the following:

* Project Overview

* System Architecture

* Principles: stakeholder feedback, transparency, accountability, iteration, stepwise action

* Activities: stand ups, back logs, sprints, sprint review and retrospectives, code reviews

* Version Control: workflow and branching model

(continued overleaf)

* Documentation: actual practices, best practice

* Tools: servers/services and involvement in choosing/setting up, backup, version control, compilers, IDEs, code documentation, user documentation, build automation, deployment, user interface prototyping.

* Quality: quality systems

* Continuous Improvement: what the group/individual could have done differently

By now, you should be able to get a sense whether they’re going to be an asset rather than a liability.

3rd Filter: Trial Period

Here’s the bad news:

☐ The mis-fits and sociopaths can last up to 6 months before being found out/the mask dropping

Have a trial period where you get them working on something of value but if it doesn’t work out, sorry - it’ll still be an expensive mistake.

TOO MUCH TECHNOLOGYTechnology is a dynamic ecosystem, with tools, languages, platforms and frameworks ever being born and dying a slow death. This is somewhat bad news for the software engineer, who is constantly pushed to adopt the unsettled new, or creakingly old, and paying the price in mental stress, time and effort.

☐ Knowing a lot about a little, and a little about a lot, will get you a long way.

☐ Adopting a new technology requires a significant investment of time and mental effort (days/months/years).

☐ To paraphrase Wayne Gretzky, “Skate to where the puck is going to be”. In the early days of the VCS wars, I picked Git because although it was new and had its problems, I felt the backing of the Linux community and the novel distributed model meant that it would offer a better return on investment in the long run over Subversion, which was mature but felt like it had peaked. 8 years later and I’m still reaping the rewards of that decision.

☐ Unfortunately, the reality is that the choice is sometimes out of your hands.

SOFTWARE ENGINEERING ACTIVITIESIt’s too common to be ideological about process… the best made plans (and designs), you know. Rather, I like to think about the activities you’d probably do and often in not the order you’d think.

Principles

☐ Continuous delivery of value

☐ Working software matters most

☐ Stuff happens

Design

Planning

Inception

Setup

Execution

INCEPTION

Why

☐ Statement of Intent

☐ Customer Benefits

☐ Slogan/Mantra

What

☐ Elevator Pitch: For [ customer ] who [ need/opportunity ] the [ product name ] is a [ product category ] that [ primary benefit ]. Unlike [ competition] our product [ differentiation ]

☐ Not To-Do List: what’s in-scope, out-of-scope or unresolved

Who

☐ Stakeholder List: people/entities involved

How

☐ System Architecture

☐ Risks

☐ Project Size: (very) rough estimates on { time, cost, effort }

☐ Potential Trade-offs: weighing tolerance of, consequences of, and potential responses to, stress on { budget, time, scope }

DESIGN

Brainstorm

☐ User Interface Mockups

☐ System Architecture

☐ Flowcharts

☐ Scenarios

☐ Storyboards

☐ Personas

Brainstorm Everything Else

☐ Constraints

☐ Performance

☐ Standards Compliance

☐ Documentation

☐ Testing

Generate

☐ Tasks

☐ User Stories

☐ Group logically

PLANNING

Prioritise

☐ Sort the tasks, stories and groups

Define Releases

☐ Choose a release numbering system

☐ Group tasks/stories into releases. Aim for them to be frequent (monthly) and well-defined.

Estimate

☐ Assign (relative, integer) effort estimates

☐ Scale points into days

☐ Translate total points into months

☐ Revisit estimate of “Project Size”

Create a Release Dashboard

☐ Create a Release Burn-Up Chart, including linear plot of estimated constant velocity

☐ Create table of Estimated Release Dates, including overhead for ‘Setup’

TOOLKIT

Whether starting from scratch or mid-development, these pages may be useful as a template for, and record of, what you learn and use in a project. They include choosing a tool for a particular purpose, making sure a document exists and instilling concepts in the group culture. Feel free to use it as a basis for your own template - every craftsman needs their own toolkit!

Process☐ Core Principles

☐ Working Guidelines

☐ Development Process

☐ BITS Management System

☐ Individual Diary

☐ Weekly Meeting

☐

☐

☐

Human Resources☐ Induction Process

☐ Interview Process

☐

☐

☐

Coding☐ Language

☐ Platforms

☐ Frameworks

☐ Compiler

☐ IDE

☐ Build System

☐ Test System

☐

☐

☐

Coding Standards☐ Code Style

☐ Code Templates

☐ Code Beautifier

☐ Directory/File Structure

☐

☐

Version Control☐ VCS

☐ VCS Server

☐ Repository Branch Structure

☐ Workflow

☐ Code Review System

☐ Commit Message Template

☐ Release Numbering System

☐

☐

☐

Standards Compliance☐ ISO 9001

☐ ISO 13485

☐ IEC 62304

☐

☐

☐

Documentation☐ Documentation - Best Practice

☐ Source Code Documentation

☐ User Documentation

☐ Document Templates

☐ Documentation Repository

☐ README☐

☐

Continuous Integration☐ Build Server

☐ Documentation Build Server

☐ Code Checking - Static Analysis

☐ Code Checking - Unit Testing

☐ Code Checking - Style

☐ Packaging

☐ Deployment

☐

☐

Design☐ System Architecture

☐ User Interface Prototypes

☐

☐

☐

IT - Installation Documentation☐ Format

☐ Installation and Configuration Process

☐ Deployment Platform

☐ Development Environment

☐ Test Environment

☐

☐

IT - Hardware☐ Displays

☐ Desktop/laptop/tablet mix

☐

IT - Backups☐ Network Backup System

☐ Backup Processes

☐ Single Sign-On

☐ Machine Management

☐

IT - Services☐ User File Shares

☐ Group File Share

☐ Group Contacts

☐ Group Calendar

☐ E-mail

☐ E-mail Signature

☐

IT - Software Assets☐ Management System

☐ Request Process

☐

☐

IT - Tools☐ Markup Standard

☐ Markup Editor

☐ WYSIWYG Editor☐ SSH client

☐ SFTP client

☐ Diagramming

☐ Mockups

☐ Password Management

☐ Regular Expression Tool

☐

☐

EXECUTION

Release Planning

* Define a release as a dynamic list of BITS

* Customers leads the direction of travel

Implementation

* Coding, Refactoring, Bug Fixing, Documenting

* Code Reviews

* Unit Testing

* Acceptance Testing

* Feedback

* On completion, record actual effort and show work productRelease Reviews

* Compare estimates to reality

* Revisit the Release Dashboard

* Open forum for issues

Visibility

* Communicate work product to stakeholders

(screenshots, letters, demo)

Weekly Meeting

* Discuss progress and active BITS

* Connect the dots between business and development motives

* Open forum for issues - process, infrastructure, pain points,

team chemistry, communication

* Look ahead to the next week

EXAMPLE TOOLKITProcess☐ Core Principles Core Principles.txt

☐ Working Guidelines Working Guidelines.txt

☐ Development Process

Development Process.pdf (see “Execution”)

☐ BITS Management System

Hansoft

☐ Individual Diary Diary.txt

%y_%m_%d %H:%M - %H-%M * Activity 1 * Activity 2

☐ Weekly Meeting Friday @ 15:00 - 17:00

Human Resources☐ Induction Process ☐ General Induction.txt

☐ Software Engineer Induction.txt

☐ Interview Process ☐ General Interview.txt ☐ Software Engineer

Interview.txt

Coding Standards☐ Code Style Qt Coding Style

☐ Code Templates Class files, C++ templates, doxygen comments

☐ Code Beautifier Clang Format

☐ Directory/File Structure

cmake, doc, include, include/cinclude, resources, scripts, src, test README, COPYRIGHT

Version Control☐ VCS Git

☐ VCS Server GitLab

☐ Repository Branch Structure

Git Flow

☐ Workflow Git Flow

☐ Code Review System GitLab Merge Requests

☐ Commit Message Template

Summary Line -blank line- Detailed description and/or - Bullet point 1 - Bullet point 2 -blank line-

☐ Release Numbering System

major.minor.revision

Coding☐ Language C++

☐ Platforms Win64, *nix

☐ Frameworks Qt, boost, OpenCV

☐ Compiler Intel

☐ IDE Qt Creator

☐ Build System CMake

☐ Test System CTest

Standards Compliance☐ ISO 9001 ✔

☐ ISO 13485 ✔

☐ IEC 62304 ✔

Documentation☐ Documentation - Best

PracticeDocumentation - Best Practice.txt

☐ Source Code Documentation doxygen

☐ User Documentation Asciidoc.

☐ Document Templates Palette, letterhead, slides

☐ Documentation Repository Hansoft Server

☐ README README.txt

Continuous Integration☐ Build Server GitLab CI

☐ Documentation Build Server doxygen invoked by Perl script

☐ Code Checking - Static Analysis

Cppcheck

☐ Code Checking - Unit Testing QTest, CTest, CDash

☐ Code Checking - Style Clang Format

☐ Packaging CPack

☐ Deployment CMake

Design☐ System Architecture System

Architecture.graffle

☐ User Interface Prototypes UI Mockups.graffle

IT - Installation Documentation☐ Format Asciidoc files

☐ Installation and Configuration Process

<path to doc repo>/IT/Install*.txt

☐ Deployment Platform XenServer, Docker

☐ Development Environment

VirtualBox, Vagrant, Puppet: packages/version/instructions

☐ Test Environment N/A

IT - Hardware☐ Displays Prefer single, large displays

☐ Desktop/laptop/tablet mix

Prefer mostly desktops, some laptops, a tablet

IT - Backups☐ Network Backup System Bacula

☐ Backup Processes Daily/Weekly/Monthly, Offsite, Tape drives

☐ Single Sign-On OpenLDAP

☐ Machine Management PuppetIT - Services☐ User File Shares Samba, <path to shares>

☐ Group File Share Samba, <path to share>

☐ Group Contacts Calendar and Contacts Server

☐ Group Calendar Calendar and Contacts Server

☐ E-mail firstname.lastname@fqdn

☐ E-mail Signature <path to doc repo>/IT/Email Signature.txt

IT - Tools☐ Markup Standard Asciidoc

☐ Markup Editor AsciidocFX

☐ WYSIWYG Editor MS Office

☐ SSH client PuTTY, OpenSSH

☐ SFTP client Cyberduck

☐ Diagramming Omnigraffle

☐ Mockup Balsamic, Omnigraffle

☐ Password Management LastPass

☐ Regular Expression Tool http://www.regexr.com

IT - Software Assets☐ Management System Snipe-IT

☐ Request Process Snipe-IT