forget me please? event sourcing & the gdpr · read cqrs / event sourcing theory followed a...

FORGET ME PLEASE?EVENT SOURCING & THE GDPR

Michiel Rook - @michieltcs

DISCLAIMER: I AM NOT A LAWYER

GENERAL DATA PROTECTION REGULATION

' Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (Data Protection Directive)

-General Data Protection Regulation

A SHORT HISTORY

1995

Data Protection Directive

1995


2012

GDPR proposal

1995


2012

GDPR proposal

2016

GDPR adopted

1995


2012

GDPR proposal

2016

GDPR adopted

25 May 2018

GDPR enforceable

REGULATION

PROTECTS EU CITIZENS

DATA PROTECTION ACT

BROAD & VAGUE

PRIVACY BY DESIGN

' The controller shall implement appropriate technical and organisational measures for ensuring that, by default, only personal data which are necessary for each specific purpose of the processing are processed. -GDPR, Article 25

DATA PROTECTION OFFICER

SUPERVISORY AUTHORITY

€20 MILLION OR 4% OF ANNUAL TURNOVER

RAISE YOUR HAND

IF YOU HAVE

read CQRS / Event Sourcing theory

RAISE YOUR HAND

IF YOU HAVE


followed a tutorial, built a hobby project

RAISE YOUR HAND

IF YOU HAVE


followed a tutorial, built a hobby project

used it in production

RAISE YOUR HAND

IF YOU HAVE

Axon Framework Spring Boot

QUICK RECAP CQRS + EVENT SOURCING

COMMAND QUERY RESPONSIBILITY

SEGREGATION

STORAGE SIDEVS.

QUERY SIDE

UI

@michieltcs

Domain

UI

Command

commands

Aggregates

@michieltcs

Domain

UI

Command

Repository

Event Store

commands

events

Aggregates

@michieltcs

Domain

UI

Event Bus

Event Handlers

Command

Repository

Database Database

Event Store

commands

events

events

Aggregates

@michieltcs

Domain

UI

Event Bus

Event Handlers

Command

Repository

Data Layer

Database Database

Event Store

commands

events

events

queries DTOs

Aggregates

@michieltcs

EVENT SOURCING

' Event Sourcing ensures that all changes to application state are stored as a sequence of events.

-Martin Fowler

ACTIVE RECORD VS. EVENT SOURCING

Account Id Account number Balance1234 12345678 �50,00

... ... ...

Money WithdrawnAccount Id 1234

Amount �50,00

Money DepositedAccount Id 1234

Amount �100,00

Account OpenedAccount Id 1234

Account number 12345678

@michieltcs

COMMANDS TO EVENTS

Deposit MoneyAccount Id 1234

Amount �100,00

@michieltcs

1 @Value 2 public class DepositMoney { 3 @TargetAggregateIdentifier 4 String accountId; 5 BigDecimal amount; 6 }

COMMANDS TO EVENTS


Amount �100,00command handler

@michieltcs

1 @CommandHandler 2 public void depositMoney(DepositMoney command) { 3 apply(new MoneyDeposited( 4 command.getAccountId(), 5 command.getAmount(), 6 ZonedDateTime.now())); 7 }

COMMANDS TO EVENTS


Amount �100,00


Amount �100,00command handler

@michieltcs

1 @Value 2 public class MoneyDeposited { 3 String accountId; 4 BigDecimal amount; 5 ZonedDateTime timestamp; 6 }

AGGREGATES

@michieltcs

an Aggregate handles Commands and generates Events based on the current state

AGGREGATES

@michieltcs

1 class BankAccount { 2 @AggregateIdentifier 3 private String accountId; 4 private String accountNumber; 5 private BigDecimal balance; 6 7 // ... 8 @EventHandler 9 public void accountOpened(AccountOpened event) { 10 this.accountId = event.getAccountId(); 11 this.accountNumber = event.getAccountNumber(); 12 this.balance = BigDecimal.valueOf(0); 13 } 14 15 @EventHandler 16 public void moneyDeposited(MoneyDeposited event) { 17 this.balance = this.balance.add(event.getAmount()); 18 } 19 }

AGGREGATE STATE

Account number Balance12345678 �0,00



event handler

event handler

event handler

@michieltcs

Money WithdrawnAccount Id 1234

Amount �50,00


Amount �100,00

Account OpenedAccount Id 1234

Account number 12345678

VALIDATING COMMANDS

@michieltcs

1 @CommandHandler 2 public void withdrawMoney(WithdrawMoney command) throws 3 OverdraftDetectedException { 4 if (balance.compareTo(command.getAmount()) >= 0) { 5 apply(new MoneyWithdrawn( 6 command.getAccountId(), 7 command.getAmount(), 8 ZonedDateTime.now())); 9 } else { 10 throw new OverdraftDetectedException(accountNumber, balance, command. 11 getAmount()); 12 } 13 }

TESTING AGGREGATES

@michieltcs

1 public class BankAccountTest { 2 private FixtureConfiguration<BankAccount> fixture; 3 4 @Before 5 public void createFixture() { 6 fixture = new AggregateTestFixture<>(BankAccount.class); 7 } 8 9 @Test 10 public void noOverdraftsOnEmptyAccount() { 11 fixture.given(new AccountOpened(ACCOUNT_ID, ACCOUNT_NUMBER)) 12 .when(new WithdrawMoney(ACCOUNT_ID, new BigDecimal(20))) 13 .expectException(OverdraftDetectedException.class); 14 } 15 16 private final static String ACCOUNT_ID = "accountId"; 17 private final static String ACCOUNT_NUMBER = "accountNumber"; 18 }

EVENT SOURCING& GDPR

CONSENT

' Where processing is based on consent, the controller shall be able to demonstrate that the data subject has consented to processing of his or her personal data.

-GDPR, Article 7

REGISTERING CONSENT

' ...the request for consent shall be presented in a manner which is clearly distinguishable from the other matters...

-GDPR, Article 7

REVOKING CONSENT

' The data subject shall have the right to withdraw his or her consent at any time. ... It shall be as easy to withdraw as to give consent.

-GDPR, Article 7

WHY USE EVENT SOURCING / CQRS?

CAPTURE INTENT

DEMONSTRATING CONSENT

EVENT LOGAS AUDIT LOG

NEW READ MODELS

EASIER DEBUGGING

EVENT LOG AS AUDIT LOG

@michieltcs

ConsentedToNewsletters


@michieltcs


ConsentedToDataGathering


@michieltcs


ConsentedToDataGathering

RevokedConsentToNewsletters

"RIGHT TO ACCESS"

' The data subject shall have the right to obtain from the controller confirmation as to whether or not personal data ... are being processed, and ... access to the personal data ...

-GDPR, Article 15

"RIGHT TO ERASURE"

' The data subject shall have the right to obtain from the controller the erasure of personal data concerning him or her without undue delay

-GDPR, Article 17

PERSONALLY IDENTIFIABLE INFORMATION

' ‘personal data’ means any information relating to an identified or identifiable natural person; an identifiable natural person is one who can be identified, directly or indirectly

-GDPR, Article 4

GROUNDS

' .. the personal data are no longer necessary .. the data subject withdraws consent on which the processing is based

-GDPR, Article 17

EXCEPTIONS

' .. to comply with a legal obligation .. for the establishment, exercise or defence of legal claims (*)

-GDPR, Article 17

UNDUE DELAY

INFORM 3RD PARTIES

BACKUPS?

PROCESSING GDPR ART. 17 REQUESTS

@michieltcs

RightToErasureInvoked


@michieltcs


Notify 3rd parties


@michieltcs


Remove from read

models

Notify 3rd parties


@michieltcs


Remove from event

store

Remove from read

models

Notify 3rd parties


@michieltcs


Remove from event

store

?

Remove from read

models

Notify 3rd parties

IMMUTABLE EVENTS?

COMPENSATING ACTIONS

@michieltcs

Ledger EntryAug 14 Inventory �15600,00

Accounts Payable �15600,00

@michieltcs





@michieltcs





Ledger Correction EntryAug 14 Inventory �900,00



class MoneyWithdrawn { String accountId; BigDecimal amount; }

class WithdrawalRolledBack { String accountId; BigDecimal amount; }

Typo: too much withdrawn!


class AccountOpened { String accountId; String accountNumber; }

class DuplicateAccountClosed { String accountId; }

Duplicate account number!

STRATEGIES

ONLY REMOVE FROM PROJECTION?

LEGAL DEFENCE?

' .. adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed (‘data minimisation’)

-GDPR, Article 5

UPCASTING?

UPCASTING

Event Store

Event_V1

Upcaster

Event_V2

Event Handler

@michieltcs

UPCASTING

Event Store

Event_V1

Upcaster

Event_V2

Event Handler

@michieltcs

Event_V2 = f(Event_V1)

DELETING EVENTS

MODIFYING EVENTS

COPY & FILTER

VERSIONED EVENT STORE


events_v1

[ { "id": "12345678", "type": "AccountOpened", "aggregateType": "Account", "aggregateIdentifier": "1234", "sequenceNumber": 0, "payloadRevision": "1.0", "payload": { ... }, "timestamp": ... ... }, ... ]

@michieltcs

COPY & REPLACE


Loop over existing events

Apply upcaster

Add queued events

Use new event store

New events Queue

@michieltcs


events_v2

[ { "id": "12345678", "type": "AccountOpened", "aggregateType": "Account", "aggregateIdentifier": "1234", "sequenceNumber": 0, "payloadRevision": "2.0", "payload": { ... }, "timestamp": ... ... }, ... ]

@michieltcs

STORE PII EXTERNALLY


@michieltcs

1 @Value 2 public class AccountOpened { 3 String accountId; 4 String accountNumber; 5 String name; 6 }


@michieltcs

1 @Value 2 public class AccountOpened { 3 String accountId; 4 }


@michieltcs

AccountOpened External Storage


Account Id Account number Name1234 12345678 John Doe

... ... ...


@michieltcs

AccountOpened External Storage


Account Id Account number Name1234 12345678 ANON

... ... ...

CRYPTO ERASURE

ENCRYPT EVENTS

DECRYPT EVENTS

ENCRYPT FIELD VALUES

DECRYPT FIELD VALUES

ENCRYPTING EVENTS

@michieltcs

<org.demo.AccountOpened> <accountId>80f49161</accountId> <accountNumberIban>NL00ABNA012345678</accountNumberIban> <firstName>Foo</firstName> <lastName>Bar</lastName> ... </org.demo.AccountOpened>

ENCRYPTING EVENTS

@michieltcs

<org.demo.AccountOpened> <accountId>80f49161</accountId> <accountNumberIban>2dqjHkY8Mc8+cek4vs/9hzgkob4J3fZJNIJh2sAXlJ0=</accountNumberIban> <firstName>N5Y27vd0UbKo6FIu5c7QGQ==</firstName> <lastName>OSKrzfuuuayuUNXYS5YUug==</lastName> ... </org.demo.AccountOpened>

ENCRYPTING EVENTS

Generate event

Find / create

encryption key

Encrypt payload values

Storeevent

@michieltcs

DECRYPTING EVENTS

Loadevent

Find associatedencryption

key

Decrypt payload values

Processevent

@michieltcs

SHEDDING THE KEY

Loadevent

Find associatedencryption

key

Decrypt payload values

Processevent

@michieltcs

X

AXON GDPR MODULE

@michieltcs

1 @Value 2 public class AccountOpened { 3 @DataSubjectId 4 String accountId; 5 6 @PersonalData 7 String accountNumberIban; 8 9 @PersonalData 10 String firstName; 11 12 @PersonalData 13 String lastName; 14 }

KEY MANAGEMENT

PERFORMANCE

RE-ENCRYPT DATA AT REST

CLOSING WORDS

CHALLENGES

FRAMEWORK SUPPORT

(IM)MUTABILITY

AUDIT TRAIL

DEMONSTRATING CONSENT

FUTURE?

THANK YOU!@michieltcs / [email protected]

www.michielrook.nl

forget me please? event sourcing & the gdpr · read cqrs / event sourcing theory followed a...

Documents