jedi mind tricks in git

Jedi Mind Tricks in GitHelp me Obi-Git, you’re my only hope

Who are we?@jankrag - @randomsort

Git Trainers

“Oh my. Software development sound rather perilous. I can assure you they will never get me

on one of those dreadful Computers”

- C3PO

this -is a command

This is the console output of a commandIt can be coloured as wellthis completed successfully.

#!/foo/barThis is the content of a script

[config]Or the content of a config filehello=git merge

Conventions

Working with git hooks

What are git hooks?

We can jam a hook into Git’s normal flow

Git hook control flow

CompleteFinalizeWrite MsgCommitStage

precommitPrepare commit msg

Commit msg postcommit

NotificationActionable

Yes - but how do I get started?git init

Making Git hooksls .git/hooks

applypatch-msg.sample commit-msg.sample post-update.sample pre-commit.sample prepare-commit-msg.sample pre-rebase.sample pre-applypatch.sample pre-push.sample update.sample

Client side vs server side hooks

precommitprepare-commit-msgcommit-msgpost-commitpost-checkoutpre-rebase

pre-receive update post-receive

The truth is out there...

Specifically on master

pre-commit hook

● First thing that happens after git commit● Able to abandon the commit● We have not got any arguments

Where are we at?git symbolic-ref --short HEAD

master

pre-commit hook#!/bin/bash# Check where our HEAD is atif [ "$(git symbolic-ref --short HEAD)" == "master" ]; then

echo "This is not the branch you are looking for"exit 1

fiexit 0

cp pre-commit .git/hooks/pre-commitgit checkout mastergit commit -am “I want to commit this”

This is not the branch you’re looking for

Well, I should be able to commit on master

git checkout mastergit commit -n -am “I want to commit this”

[master 5d5308b] I want to commit this 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 foo.md

The mission

● I want to reference an issue in my commit messages● If I do not reference an issue in my repository - abandon my commit

Where are our issues? Git remote show origin

* remote origin Fetch URL: [email protected]:RandomSort/hooks Push URL: [email protected]:RandomSort/hooks HEAD branch: master Remote branch: master tracked Local branch configured for 'git pull': master merges with remote master Local ref configured for 'git push': master pushes to master (up to date)

I <3 APIScurl -s https://api.github.com/repos/$repo/issues/$issue_number

{ "url": "https://api.github.com/repos/RandomSort/hooks/issues/1", "repository_url": "https://api.github.com/repos/RandomSort/hooks", "labels_url": "https://api.github.com/repos/RandomSort/hooks/issues/1/labels{/name}", ....TONS OF JSON :D

commit-msg hook#!/bin/bash# $1 is the temp file containing the commit messageissue_number=`cat $1 | xargs | sed 's/.*#$[0-9]*$.*/\1/'`repo="`git remote show origin | grep "Push" | xargs | sed 's/.*:$.*$/\1/'`"curl -s https://api.github.com/repos/$repo/issues/$issue_number |\ grep -q "\"Not Found\"" > /dev/nullexit $((1 -$?))

commit-msg hook#!/bin/bash# $1 is the temp file containing the commit messageissue_number=`cat $1 | xargs | sed 's/.*#$[0-9]*$.*/\1/'`repo="`git remote show origin | grep "Push" | xargs | sed \ 's/.*:$.*$/\1/'`"curl -s https://api.github.com/repos/$repo/issues/$issue_number |\ grep -q "\"Not Found\"" > /dev/nullexit $((1 -$?))

commit-msg hook#!/bin/bash# $1 is the temp file containing the commit messageissue_number=`cat $1 | xargs | sed 's/.*#$[0-9]*$.*/\1/'`repo="`git remote show origin | grep "Push" | xargs | sed 's/.*:$.*$/\1/'`"curl -s https://api.github.com/repos/$repo/issues/$issue_number |\ grep -q "\"Not Found\"" > /dev/nullexit $((1 -$?))

So now we’ve done something sane ..

● Git hooks can do useful stuff● Everything in moderation● (bust the myth, then go all in)

Trust your workflow"Fear is the path to the dark side"

Branch based delivery

C1 C2 C3 C4

ready/feature

master

HEAD

git push origin feature:ready/feature

#serverless

These are not the servers you are looking for

Going native

The mission

● I want to work on a feature branch never on master● I want to commit work referencing issues● I want to be able to run my Continuous Integration setup● I deliver through ready branches● Only allow integration if

○ Tests succeed○ it’s a fast forward merge

Where are we at (again)?git symbolic-ref --short HEAD

ready/foo

git rev-parse --show-toplevel

/home/randomsort/repos/hooks

.. And where’s our stuff?

We only want to do fast forward merges

● It’s prettiest● We prefer to have our automation engine only do trivial merges

C1 C2 C3 C4

Feature#79

master

HEAD

How fast can we go?

git rev-list --maxcount 1 master..ready/foo

C1 C2 C3 C4

ready/foo

master

HEAD

Putting it together#!bin/bashif [ not on ready branch ] do nothingfiif [ not fast forwardable ] cleanupfirun testsif [ successful ] merge cleanupelse cleanup

Cleaning up fails#We can't do a fast forward merge so let's abort this and move some stuff aroundfailbranch=`echo $branch | xargs | sed 's/ready/failed/'`echo "Unable to fast forward master to $branch leaving state on \ $failbranch"git checkout -b $failbranchgit branch -D $branch

Runninggit checkout -b ready/feature

Switched to a new branch 'ready/feature'Running testsTesting has failed, leaving state on failed/featureSwitched to a new branch 'failed/feature'Deleted branch ready/feature (was 462b135).

Running with successful testsgit checkout -b ready/feature

Switched to a new branch 'ready/feature1'Running testsSwitched to branch 'master'Updating e28500e..06a16b1Fast-forward temp.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 temp.mdDeleted branch ready/feature1 (was 06a16b1).

It works on my machine"I find your lack of faith disturbing"

TatooineJan

TATOOINE - LARS OWENS HOMESTEAD

OWEN:

I have no need for a protocol droid.

C3P0:

Sir -- not in an environment such as this -- that's why I've also been programmed for over thirty secondary functions that...

OWEN:

What I really need is a droid that understands the binary language of moisture vaporators.

... secondary functions"I have no need for a protocol droid."

Binary is annoying...

git diff wordfile.docx

diff --git a/wordfile.docx b/wordfile.docxindex d24c436..3c4aa59 100644Binary files a/wordfile.docx and b/wordfile.docx differ

No idea what changed...

Teaching git new tricks

Combining two features:

● Git attributes○ Assign behaviour per file or path

● Custom drivers○ Define new behaviour

Git attributes

Each line in an attributes definition is of form:

pattern attr1 attr2 ...

Where a attribute can be set, unset or assigned a value:

*.txt text*.jpg -text*.sh text eol=lf

Git attributes - global

Default location: $XDG_CONFIG_HOME/git/attributes. Fallback:$HOME/.config/git/attributes

If you want to set the path you can use:

git config --global core.attributesfile <path>

git config --global core.attributesfile ~/.gitattributes

e.g.

Git attributes - local

Just add definitions to a local .gitattributes file.

Can be set in root folder and in subfolders

Git attributes

commonly used for changing (force'ing):

● CRLF behaviour

● text / binary behaviour

But that is not Jedi enough ...... so we will skip those

Diff driver

Set as a config parameter like diff.foo

[diff "hexify"] binary = true textconv = hexdump -v -C

Using the new driver is easy

In .gitattributes:*.bin diff=hexify

Note - we need both:

● Driver definition in config● Path attribute that tells diff to use it

But hexdump is still pretty useless 00000000 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 |.PNG........IHDR|00000010 00 00 03 20 00 00 02 80 08 02 00 00 00 eb 79 8b |... ..........y.|00000020 65 00 00 00 09 70 48 59 73 00 00 0b 13 00 00 0b |e....pHYs.......|00000030 13 01 00 9a 9c 18 00 00 00 06 62 4b 47 44 00 ff |..........bKGD..|00000040 00 ff 00 ff a0 bd a7 93 00 00 8a b5 49 44 41 54 |............IDAT|00000050 78 da ec bd 09 5b 53 57 f7 fe df f7 55 32 9d cc |x....[SW....U2..|00000060 33 09 99 e7 79 20 09 19 50 91 41 90 41 26 41 44 |3...y ..P.A.A&AD|00000070 44 14 11 41 04 04 ad b5 b5 b6 b5 b5 56 ed ef 85 |D..A........V...|00000080 fd 17 a5 7f bf 3e 4a 4e 42 38 c9 39 27 e7 be ae |.....>JNB8.9'...|00000090 cf d5 cb c7 47 25 39 67 ef b5 ee bd f7 da f7 fa |....G%9g........|000000a0 ae a7 e7 7b 00 00 00 00 00 c0 21 df e1 11 00 00 |...{......!.....|000000b0 00 00 00 40 60 01 00 00 00 00 40 60 01 00 00 00 |...@`.....@`....|000000c0 00 40 60 01 00 00 00 00 00 08 2c 00 00 00 00 00 |.@`.......,.....|000000d0 08 2c 00 00 00 00 00 08 2c 00 00 00 00 00 00 81 |.,......,.......|000000e0 05 00 00 00 00 00 81 05 00 00 00 00 00 81 05 00 |................|000000f0 00 00 00 00 20 b0 00 00 00 00 00 20 b0 00 00 00 |.... ...... ....|00000100 00 00 20 b0 00 00 00 00 00 20 b0 00 00 00 00 00 |.. ...... ......|00000110 00 04 16 00 00 00 00 00 04 16 00 00 00 00 00 04 |................|<pages and pages without end>

Recipe for success

For each binary type:

1. Find a tool that extracts useful info from a file2. Set up a diff driver to use it3. Profit

4. Don't give in to the dark side

Word files

My brain parses markdown quite fine...

*.docx diff=pandoc2md

[diff "pandoc2md"]textconv=pandoc --to=markdown

brew install pandoc

For reference:

.gitattributes

config

git diff wordfile.docx

diff --git a/wordfile.docx b/wordfile.docxindex d24c436..48249e3 100644--- a/wordfile.docx+++ b/wordfile.docx@@ -1,8 +1,8 @@-Headline+Hi Git Merge This is some simple text -*and another important paragraph in italic*+*and another* **important** *paragraph in italic*

Word to MarkDown

git diff --word-diff=color wordfile.docx

diff --git a/wordfile.docx b/wordfile.docxindex d24c436..48249e3 100644--- a/wordfile.docx+++ b/wordfile.docx@@ -1,8 +1,8 @@HeadlineHi Git Merge

This is some simple text

*and another* **important** *paragraph in italic*

Word to MarkDown

Other ideas?

PDF files*.pdf diff=pdfconv

[diff "pdfconv"]textconv=pdftohtml -stdout

brew install pdftohtml

<BODY bgcolor="#A0A0A0" vlink="blue" link="blue"> <A name=1></a>Headline This is some simple text -and another paragraph -and stuff +and another important paragraph in italic +stuff goes here +Greetings from Denmark <hr> </BODY>

Maybe even for non-binaries?

Syntax highlighting

*.py diff=color

pip install Pygments

For reference:

.gitattributes

config

[diff "color"]textconv=pygmentize

git diff --cached primes.py

...@@ -0,0 +1,22 @@+def primes(n):+ """+ Compute the sequence of the first n primes.+ """+ p = []+ if n > 0:+ p = [2, ]+ for x in range(1, n):+ q = p[-1] + 1+ while True:+ for y in p:+ if not (q % y):+ break+ else:+ p.append(q)

Syntax highlighted code in diff'sMy brain parses coloured python better

Snippet from:https://github.com/cbcunc/primer (GPL)

https://github.com/cbcunc/primer

https://github.com/cbcunc/primer

Reformatting

*.md diff=wrap

brew install fmt

For reference:

.gitattributes

config

[diff "wrap"]textconv=fmt

git diff autostash.md

>"Why can't I pull when I have a dirty workspace, when Mercurial can do this out of the box?" -I gave the immediate answer that this is just Git's way of protecting the user from possibly harmful and, more importantly, irreversible changes. Git by default takes a very paranoid approach to any operations that change dirty files in your file system, when Git itself can't get you out of those changes again. _This is normally considered a feature_. The known "workaround", or possible workflow, is to stash any changes before doing a pull (with `git stash save`, and then unstash them again (`git stash pop`) when done. It seems obvious that it should be easy to automate this with a git alias, but it turns out that this isn't trivial, as git stash doesn't fail gracefully when there are no local changes.+I gave the immediate answer that this is just Git's way of protecting the user from possibly harmful and, more importantly, irreversible changes. Git by default takes a very paranoid approach to any operations that change dirty files in your file system, when **Git** itself can't get you out of those changes again. _This is normally considered a feature_. The known "workaround", or possible workflow, is to stash any changes before doing a pull (with `git stash save`, and then unstash them again (`git stash pop`) when done. It seems obvious that it should be easy to automate this with a git alias but it turns out that this isn't trivial. Git stash does not fail gracefully when there are no local changes.

.md - Before reformatting

git diff autostash.md

@@ -13,13 +13,13 @@ can do this out of the box?" I gave the immediate answer that this is just Git's way of protecting the user from possibly harmful and, more importantly, irreversible changes. Git by default takes a very paranoid approach to any-operations that change dirty files in your file system, when Git+operations that change dirty files in your file system, when **Git** itself can't get you out of those changes again. _This is normally considered a feature_. The known "workaround", or possible workflow, is to stash any changes before doing a pull (with `git stash save`, and then unstash them again (`git stash pop`) when done. It seems-obvious that it should be easy to automate this with a git alias,-but it turns out that this isn't trivial, as git stash doesn't fail+obvious that it should be easy to automate this with a git alias+but it turns out that this isn't trivial. Git stash does not fail gracefully when there are no local changes.

.md - after formatting with fmt

Images

Extract meta data

Define a useful driver:[diff "exif"]

textconv=exiftool

*.jpg diff=exif*.jpeg diff=exif*.png diff=exif*.gif diff=exif

https://openclipart.org/detail/227445/yoda

Let's resize Yoda

git diff Yoda-800px.png

diff --git a/Yoda-800px.png b/Yoda-800px.pngindex fc8ee0f..8ed738c 100644--- a/Yoda-800px.png+++ b/Yoda-800px.png@@ -1,24 +1,26 @@-File Size : 35 kB-File Modification Date/Time : 2017:01:27 17:29:06+01:00+File Size : 85 kB+File Modification Date/Time : 2017:01:30 18:52:20+01:00 File Type : PNG-Image Width : 800-Image Height : 640+Image Width : 600+Image Height : 480 Bit Depth : 8-Color Type : RGB+Color Type : RGB with Alpha

Much more useful

Note: manually abbreviated for slide

But do I know what changed?

Can we see the difference?

cachetextconv caches the result

[diff "jpg"]textconv=jp2a --width=80cachetextconv = true

*.jpg diff=jpg*.jpeg diff=jpg

Can't quite do holograms yet, but...

brew install jp2a

git add Yoda.jpg

git diff --cached

diff --git a/Yoda.jpg b/Yoda.jpgnew file mode 100644index 0000000..cf0300a--- /dev/null+++ b/Yoda.jpg@@ -0,0 +1,32 @@+................................................................................+......................................''''......................................+................................,:ldxkkOOkkxdl:,................................+.............................,lxOOdlllxOOxllldOkxc'.............................+....',:ccllllollc:,'.......'lkOxllllllxOOxllllllkOkl'.......',:cllooollcc:,'....+..;okOOkccccclllodxkxoc;'.:xOOOkkxddddkOOkodddxkkOOOx;.';coxkxdollccccclkOOko;..+....':dOxl;;:::::;;;:ldx:oOxolllloddxxkOOkxxddollllokklcxdl:;;;:::::;:okkd:'....+.......;xOkd:;::::::::':xOxoxOOOOOOOOOOOOOOOOOOOOOOxokOx;,::::::::;:xOOx;.......+........'okOOd;;:::::,okOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOkl,:::::;;dOOkl'........+..........;dOOkl;;:;:xOOOOOOOkkOOOOOOOOOOOOOOOOOOkkOOOOOOOx;;:;;okOkd,..........+............;okOkd,ckOOOOOkc,..,oOOOOOOOOOOOOOkl,..,lkOOOOOk;;dkOkl,............+..............';lo;kOOOOOOl......xOOOOOOOOOOOOd......oOOOOOOk;dl;...............+..................lOOOOOOOk;...'ckOOOOOOOOOOOOk:'...:kOOOOOOOc..................+..................xOOOOOOOOOxddkOOOOOOOOOOOOOOOOkddkOOOOOOOOOo..................

https://openclipart.org/detail/227445/yoda

Let's give him eyes

diff --git a/Yoda.jpg b/Yoda.jpgindex cf0300a..c16b7de 100644--- a/Yoda.jpg+++ b/Yoda.jpg@@ -9,8 +9,8 @@ ........'okOOd;;:::::,okOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOkl,:::::;;dOOkl'........ ..........;dOOkl;;:;:xOOOOOOOkkOOOOOOOOOOOOOOOOOOkkOOOOOOOx;;:;;okOkd,.......... ............;okOkd,ckOOOOOkc,..,oOOOOOOOOOOOOOkl,..,lkOOOOOk;;dkOkl,............-..............';lo;kOOOOOOl......xOOOOOOOOOOOOd......oOOOOOOk;dl;...............-..................lOOOOOOOk;...'ckOOOOOOOOOOOOk:'...:kOOOOOOOc..................+..............';lo;kOOOOOOl'oxc..xOOOOOOOOOOOOd,dd:..oOOOOOOk;dl;...............+..................lOOOOOOOk:odc'ckOOOOOOOOOOOOkldd:.:kOOOOOOOc.................. ..................xOOOOOOOOOxddkOOOOOOOOOOOOOOOOkddkOOOOOOOOOo.................. ..................dOOOOOOOOOOOOOkookOOOOOOOOOdldOOOOOOOOOOOOOl.................. ..................,kOOOOOOOOOOkcc:oxkOOOOOOkxocccxOOOOOOOOOOx'..................

git diff Yoda.jpg

Even in gitk

Other quick onesMP3 file meta data: exiftool can do those too, or try mp3info:

[diff "mp3"]textconv=mp3info -x

Excel files:

[diff "xlsconv"]textconv=xls2csv

Zip files:

[diff "zipshow"]textconv=unzip -c -a

[diff "ziplist"]textconv=unzip -l

Length Date Time Name --------- ---------- ----- ---- 0 01-25-2017 12:52 stuff/ 0 01-25-2017 12:53 stuff/foo/ 7 01-25-2017 12:53 stuff/foo/bar 12 01-25-2017 12:52 stuff/hello.world- 9 01-25-2017 12:52 stuff/hi.txt --------- -------- 28 5 files+ 19 4 files

git diff stuff.zip

And we haven't even left Tatooine yet!

Recap - Git objects

https://github.com/pluralsight/git-internals-pdf

Filter driversProcess blobs on save or checkout

[filter "name"]clean = <command>smudge = <command>

Let's have some fun with them too...

There is no try...

Yapf = Yet Another Python Formatter( pip install --user yapf )

[filter "cleanpython"]clean = yapf smudge = cat

*.py filter=cleanpython

ugly.py x = { 'a':37,'b':42,

'c':927}

y = 'hello ''world'z = 'hello '+'world'a = 'hello {}'.format('world' )class foo ( object ): def f (self ): return 37*-+2 def g(self, x,y=42): return ydef f ( a ) : return 37+-+a[42-x : y**3]

git add ugly.py

git commit ....

git push

... and behold.

Caveat

That was a conceptual demo - probably not production ready.

Git diff gets a bit confused:

● git status now shows ugly.py as modified● git add - and it disappears :-)

“Your eyes can deceive you. Don’t trust them.” – Obi-Wan

For demo: simple rot13 'encryption' - only affects letters

[filter "secret"]clean = ruby -ne 'print $_.tr( \"A-Za-z\", \"N-ZA-Mn-za-m\") ' smudge = perl -pe 'tr/N-ZA-Mn-za-m/A-Za-z/'

*.java filter=secret

public static void sort(int[] numbers) { int n = numbers.length; int temp = 0;

for (int i = 0; i < n; i++) { for (int j = 1; j < (n - i); j++) {

if (numbers[j - 1] > numbers[j]) { //comment temp = numbers[j - 1]; numbers[j - 1] = numbers[j]; numbers[j] = temp; } } }}

sort.java

git ls-tree HEAD

100644 blob cf0300a887b362db6e26891afb6ad86757e00f72 Yoda.jpg100644 blob 60f95e16c59e6e0ef7dc5542cc0f5c15a35e9df4 primes.py100644 blob 887177cdd3976c695ec5303e6a6d2c1832b458a9 sort.java100644 blob 809ec4f968232287368681257a0b5ad5c726c08a ugly.py

git show 887177cdd

choyvp fgngvp ibvq fbeg(vag[] ahzoref) { vag a = ahzoref.yratgu; vag grzc = 0;

sbe (vag v = 0; v < a; v++) { sbe (vag w = 1; w < (a - v); w++) {

vs (ahzoref[w - 1] > ahzoref[w]) { grzc = ahzoref[w - 1]; ahzoref[w - 1] = ahzoref[w]; ahzoref[w] = grzc; }

} }}

And on GitHub too, obviously

Now just imagine we had used

● GPG or AES-256 with private key● *.cred filter=secret

Publish your private content to public repos :-)

The End"Already know you that which you need." – Yoda

jedi mind tricks in git

Technology