postgresql day italy 2016 unit test

Andrea Adami ([email protected])

PGDAY 2016

1

PostgreSQL DAY2016

UNITTESTING


PGDAY 2016

2

PostgreSQL DAY2016

BUONGIORNOE

BENVENUTI


PGDAY 2016

3

PostgreSQL DAY2016

ANDREA ADAMI


PGDAY 2016

4

who am i Mi definisco un Solution Architect

Ho cominciato a interessarmi di informatica quando ho trovato sotto l'albero di natale del 1980 un Commodore 64.

L'enorme quantità di ram (64k) rispetto al precedente VIC20 (5k) mi entusiasma e mi spinge a muovere primi passi nella programmazione fino a farne la mia professione nel 1985.

Il mio interesse ora è nettamente orientato verso il mondo del software libero, e la mia curiosità è attratta dal mondo della Computer Aided Software Engineering e dalle metodologie Agili.

Nel mio tempo libero mi appassiona costruire e far volare aeromodelli.


PGDAY 2016

5

who am i Membro ATTIVO del Linux User Group di verona (http: //www.lugverona.it)

Nel direttivo del Verona FABLAB (http:/ /www.veronafablab.it)

l inkedin: https: / / it . l inkedin.com/in/andreaadami/it

Google+: https:/ /plus.google.com/+AndreaAdamiProfi le

Instagram: https: //www.instagram.com/folvr

Twitter: https:/ / tw itter.com/folstuff

http://www.veronafablab.it/

https://it.linkedin.com/in/andreaadami/it

https://plus.google.com/+AndreaAdamiProfile

https://www.instagram.com/folvr/

https://twitter.com/folstuff


PGDAY 2016

6

GRAZIE SPONSOR !

http://www.miriade.it/

http://interlogica.it/en/

https://2ndquadrant.com/it/

http://www.grusp.org/it/

http://www.esosphera.com/


PGDAY 2016

7

DEFINIZIONI

https://it .wikipedia.org/wiki/Unit_testing

In ingegneria del software, per unit testing (testing d'unita o testing unitario) si intende l 'att ivita di testing ’ ’(prova, col laudo) di singole unita software. Per unita si ’ ’intende normalmente i l minimo componente di un programma dotato di funzionamento autonomo; a seconda del paradigma di programmazione o l inguaggio di programmazione, questo puo ’corrispondere per esempio a una singola funzione nel la programmazione procedurale, o una singola classe o un singolo metodo nella programmazione a oggetti .

https://it.wikipedia.org/wiki/Unit_testing


PGDAY 2016

8

COSTI / BENEFICI

● COSTI– INERZIA– CODING

● BENEFICI– STABILITA– TEMPO

● al l inizio di piu’ ’

● poi si recupera


PGDAY 2016

9

AMBIENTE DI PROVA

Virtual Box per virtualizzare

https://www.virtualbox.org/wiki/Downloads

https://www.virtualbox.org/wiki/Downloads


PGDAY 2016

10

AMBIENTE DI PROVA

Macchina virtuale con ubuntu 16.04.1 LTS

https://www.ubuntu.com/download/server

https://www.ubuntu.com/download/server


PGDAY 2016

11

AMBIENTE DI PROVA

Database da GITHUB Scuola247

https://github.com/Scuola247/PostgreSQLREADME.md : istruz ion i per (Virtua lbox/Ubuntu/PostgreSQL/Scuola247)

https://github.com/Scuola247/PostgreSQL


PGDAY 2016

12

AMBIENTE DI PROVA

Scuola247 http://www.scuola247.orgREADME.md : istruz ion i per (Virtua lbox/Ubuntu/PostgreSQL/Scuola247)

http://www.scuola247.org/


PGDAY 2016

13

AMBIENTE DI PROVA

MIGLIAIA di r ighe d i dat i

MIGLIAIA di photo prese da Wik imedia Commons:https:/ /commons.wik imedia .org/wik i /Main_Pagecon documentaz ione del la l icenza per i l l ibero ut l izzo

8 schemi

64 tabel le

69 v iste

32 tr igger

87 funz ion i

https://commons.wikimedia.org/wiki/Main_Page


PGDAY 2016

14

UNITA’ SOFTWARE

In Postgres cos e una singola unita software ?’ ’ ’


PGDAY 2016

15

UNITA’ SOFTWARE

I domini di dati (Domain)

CREATE DOMAIN utility.week_day

AS smallint

CONSTRAINT week_day_range CHECK (VALUE >= 1 AND VALUE <= 7);


PGDAY 2016

16

UNITA’ SOFTWARE

I t ipi di dati (Type)

CREATE TYPE public.sex AS ENUM

('M',

'F');


PGDAY 2016

17

UNITA’ SOFTWARE

Per chi ama la precisione:

CREATE TYPE public.sex AS ENUM

('M',

'F',

'?',

'H',

'SH',

'MC',

'FC',

'HM',

'HF',

'HT',

'I',

'A',

'X',

'O',

'MP',

'FP',

'CP');


PGDAY 2016

18

UNITA’ SOFTWARE

https://de.wikipedia.org/wiki/Datenstandards_zur_Beschreibung_des_Geschlechts




PGDAY 2016

19

UNITA’ SOFTWARE

Conversioni (Cast)

CREATE CAST (mime_type AS file_extension)

WITH FUNCTION public.file_extension(mime_type)

AS IMPLICIT;

CREATE CAST (file_extension AS mime_type)

WITH FUNCTION public.mime_type(file_extension)

AS IMPLICIT;


PGDAY 2016

20

UNITA’ SOFTWARE

Le funzioni (Function)CREATE FUNCTION utility.day_name(_weekday utility.week_day)

RETURNS character varying AS

$BODY$

<<me>>

DECLARE

function_name varchar = 'dayname';

BEGIN

RETURN to_char('2000-01-02'::date + _weekday, 'TMDay');

END;

$BODY$


PGDAY 2016

21

UNITA’ SOFTWARE

Le funzioni (Function)CREATE FUNCTION utility.day_name(_weekday utility.week_day)


$BODY$

<<me>>

DECLARE

function_name varchar = 'dayname';

BEGIN

RETURN to_char('2000-01-02'::date + _weekday, 'TMDay');

END;

$BODY$


PGDAY 2016

22

UNITA’ SOFTWARE

Le funzioni (Function)CREATE OR REPLACE FUNCTION public.italian_fiscal_code(

name character varying,

surname character varying,

sex sex,

birthday date,

country_of_birth smallint,

city_of_birth character varying)


$BODY$


PGDAY 2016

23

UNITA’ SOFTWARE

Le viste (Views)CREATE OR REPLACE VIEW public.valutations_stats_classrooms_students_subjects AS SELECT va.classroom, va.student, va.subject, min(vo.thousandths) AS min, max(vo.thousandths) AS max, round(avg(vo.thousandths)) AS media, round(stddev_pop(vo.thousandths)) AS dev_std FROM valutations va JOIN grades vo ON vo.grade = va.grade GROUP BY va.classroom, va.student, va.subject;


PGDAY 2016

24

UNITA’ SOFTWAREL e v i s t e ( V i e w s ) c o n w i n d o w f u n c t i o n e j o i n m u l t i p l iC R E A T E V I E W p u b l i c . v a l u t a t i o n s _ r e f e r e n c e s A S S E L E C T t . c l a s s r o o m , t . t e a c h e r , t . s u b j e c t , t . o n _ d a t e , t . g r a d e _ t y p e , t . g r a d e _ t y p e _ d e s c r i p t i o n , t . g r a d e _ t y p e _ m n e m o n i c , t . t o p i c , t . t o p i c _ d e s c r i p t i o n , t . m e t r i c , t . m e t r i c _ d e s c r i p t i o n , r o w _ n u m b e r ( ) O V E R ( P A R T I T I O N B Y t . c l a s s r o o m , t . t e a c h e r , t . s u b j e c t , t . o n _ d a t e O R D E R B Y t . g r a d e _ t y p e _ d e s c r i p t i o n , t . t o p i c _ d e s c r i p t i o n , t . m e t r i c _ d e s c r i p t i o n ) A S r o w _ n u m b e r F R O M ( S E L E C T D I S T I N C T v a . c l a s s r o o m , v a . t e a c h e r , v a . s u b j e c t , v a . o n _ d a t e , v a . g r a d e _ t y p e , t v . d e s c r i p t i o n A S g r a d e _ t y p e _ d e s c r i p t i o n , t v . m n e m o n i c A S g r a d e _ t y p e _ m n e m o n i c , v a . t o p i c , C O A L E S C E ( a . d e s c r i p t i o n , ' ' : : c h a r a c t e r v a r y i n g ) A S t o p i c _ d e s c r i p t i o n , m . m e t r i c , m . d e s c r i p t i o n A S m e t r i c _ d e s c r i p t i o n F R O M v a l u t a t i o n s v a L E F T J O I N t o p i c s a O N a . t o p i c = v a . t o p i c J O I N g r a d e _ t y p e s t v O N t v . g r a d e _ t y p e = v a . g r a d e _ t y p e J O I N g r a d e s v o O N v o . g r a d e = v a . g r a d e J O I N m e t r i c s m O N m . m e t r i c = v o . m e t r i c O R D E R B Y v a . o n _ d a t e , t v . d e s c r i p t i o n , ( C O A L E S C E ( a . d e s c r i p t i o n , ' ' : : c h a r a c t e r v a r y i n g ) ) , m . d e s c r i p t i o n ) t ;


PGDAY 2016

25

UNITA’ SOFTWARE

Le v iste (Views) con crosstab funct ionCREATE OR REPLACE VIEW unit_testing.tests_check_points_count_crosstab AS SELECT crosstab.test, COALESCE(crosstab."Failed", 0) AS "Failed", COALESCE(crosstab."Passed", 0) AS "Passed", COALESCE(crosstab."Skipped", 0) AS "Skipped", COALESCE(crosstab."Failed", 0) + COALESCE(crosstab."Passed", 0) + COALESCE(crosstab."Skipped", 0) AS "Total" FROM crosstab('SELECT test, status, count FROM unit_testing.tests_check_points_count ORDER BY 1'::text, 'SELECT enum_value FROM utility.enums_values WHERE schema_name = ''unit_testing'' AND enum_name = ''check_point_status'' ORDER BY 1'::text) crosstab(test integer, "Failed" integer, "Passed" integer, "Skipped" integer);


PGDAY 2016

26

UNITA’ SOFTWARE

Le tabelle (Tables)

● CONSTRAINT NULL (NOT NULL)

● CONSTRAINT DEFAULT

● CONSTRAINT UNIQUE

● CONSTRAINT CHECK

● CONSTRAINT REFERENCES

● EXCLUDE


PGDAY 2016

27

CREATE TABLE public.school_years

(

school_year bigint NOT NULL DEFAULT nextval('pk_seq'::regclass),

school bigint NOT NULL,

description character varying(160) NOT NULL,

duration daterange,

lessons_duration daterange,

CONSTRAINT school_years_pk PRIMARY KEY (school_year),

CONSTRAINT school_years_fk_school FOREIGN KEY (school)

REFERENCES public.schools (school) MATCH SIMPLE

ON UPDATE CASCADE ON DELETE RESTRICT,

CONSTRAINT school_years_ex_duration EXCLUDE

USING gist (school WITH =, duration WITH &&), -- in the same school we cannot have duration overlap

CONSTRAINT school_years_uq_description UNIQUE (school, description), -- description must be unique

CONSTRAINT school_years_ck_duration CHECK (duration @> lessons_duration)

)

UNITA’ SOFTWARE


PGDAY 2016

28

UNITA’ SOFTWARE

I tr igger (Always Function)IF new.behavior IS NOT NULL THEN

IF (TG_OP = 'UPDATE') THEN

-- check that subject's school as equal as school

PERFORM 1 FROM subjects WHERE subject = new.behavior

AND school = new.school;

IF NOT FOUND THEN

RAISE EXCEPTION USING ......

END IF;

ELSE

--

-- cannot set the behavior because it needs school. You must:

-- 1) insert school

-- 2) insert subject

-- 3) update school with the subject

--

RAISE EXCEPTION USING ...

END IF;

END IF;

RETURN NEW;


PGDAY 2016

29

COSA TESTARE ?

● Attenzione a non testare PostgreSQL● Testate le cose piu diffici l i’

– TRIGGER– FUNCTION– FOREIGN KEY

● Poi se avete tempo e voglia– Domains– Types


PGDAY 2016

30

DIFFERENZE CON ALTRI LINGUAGGI

POCHE

TEMPO DEDICATO PROPORZIONALMENTE UGUALE TENDENTE A DIMINUIRE PER

1 GRANDE VANTAGGIO


PGDAY 2016

31

1 GRANDE VANTAGGIO

NON ABBIAMO BISOGNO DI .. .

MOCK OBJECT


PGDAY 2016

32

MOCK OBJECT

https://en.wikipedia.org/wiki/Mock_object

In object-oriented programming, mock objects are simulated objects that mimic the behavior of real objects in controlled ways. A programmer typically creates a mock object to test the behavior of some other object, in much the same way that a car designer uses a crash test dummy to simulate the dynamic behavior of a human in vehicle impacts.

https://en.wikipedia.org/wiki/Mock_object


PGDAY 2016

33

1 GRANDE VANTAGGIO

ABBIAMO LE TRANSAZIONI


PGDAY 2016

34

1 GRANDE VANTAGGIO

BEGIN test1 test2 tes3 testn RAISE EXCEPTION SQLSTATE 'ZZZZZ';EXCEPTION WHEN SQLSTATE 'ZZZZZ'END;


PGDAY 2016

35

TEST FRAMEWORKS

https://wiki.postgresql.org/wiki/Test_Frameworks

Epic

pgTap

PGUnit

Simple pgunit

https://wiki.postgresql.org/wiki/Test_Frameworks


PGDAY 2016

36

PLPGUNIT

Un grazie a Binod Nirvan:

https://np.l inkedin.com/in/binodnirvan

https://github.com/mixerp/plpgunit

https://np.linkedin.com/in/binodnirvan

https://github.com/mixerp/plpgunit


PGDAY 2016

37

PLPGUNIT

Problemi:● I l comando di lancio dei test non usa le

transazioni● Si puo racchiudere i l comando in una ’

transazione ma anche i dati risultanti dai test vengono ripristinati (quindi persi)

● Poco dettaglio in caso di errore non previsto● Poco dettaglio del singolo test


PGDAY 2016

38

UNIT_TESTING(il framework)

● Ogni test fatto e un checkpoint’

● I checkpoint sono raggruppati in function di unit test

● Le unit test function hanno una caratteristica fondamentale:ritornano: array di unit_test_result

● I l framework automaticamente gestisce il richiamo di tutte le unit_test functions in un unica transazione che viene SEMPRE abortita


PGDAY 2016

39


● Si possono dichiarare dipendenze fra unit_tests usando la tabella dependencies

● I l framework poi si occupa di richiamare nell ordine dichiarato le singole unit test’

● Se una unit test fall isce i l framework automaticamente salta tutte le unit test che dipendono da quella fall ita

● I l framework controlla eventuali riferimenti circolari infinit i


PGDAY 2016

40


*Si possono costituire unit test set per limitare il richiamo di unit test function (comodo durante lo sviluppo di una funzione per limitare i messaggi informativi)

* da sviluppare


PGDAY 2016

41


● SCHEMAS– assert– diagnostic– unit_testing– unit_tests– uti l ity


PGDAY 2016

42


La magia e :’

SELECT unit_testing.run()


PGDAY 2016

43


Sintassi completa:

unit_testing.run(

IN _check_functions boolean DEFAULT FALSE,

IN _check_queries boolean DEFAULT FALSE,

IN _check_unit_tests boolean DEFAULT TRUE,

IN _unit_test_set bigint DEFAULT NULL::bigint,

IN _verbosity text DEFAULT 'notice': :text,

IN _note text DEFAULT NULL::text,

OUT _current_test bigint)


PGDAY 2016

44


parametro:

check_functions = TRUE / FALSE

equivale all’ esecuzione o meno di:

SELECT * FROM diagnostic.functions_check;


PGDAY 2016

45


https://github.com/okbob/plpgsql_check

https://github.com/okbob/plpgsql_check


PGDAY 2016

46


parametro:

check_queries = TRUE / FALSE

equivale all’ esecuzione o meno di:

SELECT * FROM diagnostic.diagnostic.views_working;


PGDAY 2016

47


parametro:

check_unit_tests = TRUE / FALSE

se FALSE non esegue alcuna test, se TRUE si ’


PGDAY 2016

48


parametro:

*unit_test_set = <unit test set>

l imita i test eseguit a quel l i e lencati nel la unit_test_set indicata e relat ive dipendenze

*da svi luppare


PGDAY 2016

49


parametro:

verbosity = <verbosity level>

equiva le a l l’ esecuzione d i :

SET CLIENT_MIN_MESSAGES TO <verbosity level>

Controls which message levels are sent to the client. Valid values are DEBUG5, DEBUG4, DEBUG3, DEBUG2, DEBUG1, LOG, NOTICE, WARNING, ERROR, FATAL, and PANIC. Each level includes all the levels that follow it. The later the level, the fewer messages are sent. The default is NOTICE. Note that LOG has a different rank here than in log_min_messages.


PGDAY 2016

50


I l problema del sovraffollamento diagnostico


PGDAY 2016

51


parametro:

note = <note text>

Riporta i l valore del parametro nel la colonna noe del la tabel la ‘ ’unit_tests dove vengono memorizzat i tutt i i test eseguit


PGDAY 2016

52


Risultato:

NOTIFICA: --------------------------------

NOTIFICA: ---------- STATISTICS ----------

NOTIFICA: --------------------------------

NOTIFICA: Test id.........................: 773

NOTIFICA: Test started on.................: 2016-12-13 11:45:03.381946

NOTIFICA: Test ended on...................: 2016-12-13 11:45:03.409572

NOTIFICA: Test runtime was................: 00:00:00.027626

NOTIFICA: total functions............: 3

NOTIFICA: functions passed...........: 0

NOTIFICA: functions failed...........: 1

NOTIFICA: functions skipped..........: 2

NOTIFICA: total check points....: 4

NOTIFICA: check points passed...: 2

NOTIFICA: check points failed...: 2


PGDAY 2016

53


DEMO


PGDAY 2016

54

CONTINOUS INTEGRATION

https://en.wikipedia.org/wiki/Continuous_integration

In software engineering, continuous integration (CI) is the practice of merging all developer working copies to a shared mainline several times a day. Grady Booch first named and proposed CI in his 1991 method, although he did not advocate integrating several times a day. Extreme programming (XP) adopted the concept of CI and did advocate integrating more than once per day - perhaps as many as tens of times per day.

https://en.wikipedia.org/wiki/Continuous_integration


PGDAY 2016

55


CREATE EVENT TRIGGER continuous_integration ON ddl_command_end

WHEN TAG IN ('ALTER FUNCTION')

EXECUTE PROCEDURE unit_testing.tr_continuous_integration();


PGDAY 2016

56


CREATE EVENT TRIGGER cont inuous_integrat ion

ON ddl_command_end

WHEN TAG IN ( 'ALTER FUNCTION')

EXECUTE PROCEDURE unit_test ing .tr_cont inuous_integrat ion() ;


PGDAY 2016

57

FUTURO ?

● Implementare la gestione di unit_test_set● Arricchire lo schema assert con nuove

funzioni dedicate a semplif icare l uso del ’framework


PGDAY 2016

58

CONTENUTI

Tutto quanto avete visto oggi sara su:’

Slide:

http://www.slideshare.net/andreaadami

Codice:

https://github.com/Scuola247/PostgreSQL(comprese le istruz ion i per r iprodurre tutto l ambiente oggi usato)’

http://www.slideshare.net/andreaadami

https://github.com/Scuola247/PostgreSQL


PGDAY 2016

59

Q & A

Domande e ... si spera ...... Risposte


PGDAY 2016

60

GRAZIE PER L ATTENZIONE’

Quest opera e da att r ibu i re a :’ ’

Andrea Adami (ht tp : / /www.fo ls tuf f .eu)

Eccetto quando d iversamente ind icato quest opera e l icenz ia ta con la l icenza :’ ’

Creat ive Commons Attr ibuz ione - Cond iv id i a l lo stesso modo 4.0 Internaz iona le

http : / /creat ivecommons .org/ l icenses/by-sa/4.0 ( testo completo de l la l icenza)

http://www.folstuff.eu/

http://creativecommons.org/licenses/by-sa/4.0

postgresql day italy 2016 unit test

Technology