death of the batch job
TRANSCRIPT
![Page 1: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/1.jpg)
Dennis van der Stelt
all your batch jobs are belong to us
Dennis van der Stelt
Software Architect
http://dennis.bloggingabout.net/
Particular Software engineer
death of the batch job
@dvdstelt
#nservicebus
![Page 2: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/2.jpg)
Dennis van der Stelt
AGENDA
![Page 3: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/3.jpg)
Dennis van der Stelt
NServiceBusIn Particular
![Page 4: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/4.jpg)
Dennis van der Stelt
NServiceBus
It’s not like WCF, which does RPC
But closer to WCF than to BizTalk
![Page 5: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/5.jpg)
Dennis van der Stelt
BUS TOPOLOGY
![Page 6: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/6.jpg)
Dennis van der Stelt
MessagingWhat is it and why do I need it?
![Page 7: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/7.jpg)
Dennis van der Stelt
SpatialTemporalPlatform
coupling aspects
![Page 8: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/8.jpg)
demo
Quick NServiceBus Demo
![Page 9: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/9.jpg)
Dennis van der Stelt
NServiceBus SagasA pattern by relation database community
![Page 10: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/10.jpg)
Dennis van der Stelt
![Page 11: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/11.jpg)
Dennis van der Stelt
PROCESS MANAGER
process managerInitiating message
![Page 12: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/12.jpg)
demo
NServiceBus Sagas
![Page 13: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/13.jpg)
Dennis van der Stelt
Sagas Recap“What have you done” – Within Temptation
![Page 14: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/14.jpg)
Dennis van der Stelt
HANDLING MESSAGES
Behavior like normal message handlers
class MySaga : IHandleMessages<MyMessage>{public void Handle(MyMessage message){…
}}
![Page 15: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/15.jpg)
Dennis van der Stelt
STARTING SAGAS
Extends the default IHandleMessages<T>
class MySaga : IAmStartedByMessages<MyMessage>{public void Handle(MyMessage message){…
}}
![Page 16: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/16.jpg)
Dennis van der Stelt
STORING STATE
Extends the default IHandleMessages<T>
class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>
{public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;
}}
![Page 17: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/17.jpg)
Dennis van der Stelt
CORRELATING MESSAGES TO SAGA INSTANCE
class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>
{protected override void ConfigureHowToFindSaga(SagaPropertyMapper<MySagaData> mapper){mapper.ConfigureMapping<MyMessage>(m => m.MyProperty).ToSaga(s => s.MyStateProperty);
}
public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;
}}
![Page 18: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/18.jpg)
Dennis van der Stelt
REQUESTING TIMEOUTS
Reminders to the Saga itself
class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>,IHandleTimeouts<MyTimeout>
{public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;RequestTimeout<MyTimeout>(TimeSpan.FromSeconds(10));
}
public void Timeout(MyTimeout state){…
}}
![Page 19: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/19.jpg)
Dennis van der Stelt
SENDING MESSAGES
Reminders to the Saga itself
class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>
{public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;
this.Bus.Send(new MyCommand());this.Bus.Publish(new MyEvent());
}}
![Page 20: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/20.jpg)
Dennis van der Stelt
SAGA STATE
Memento
class MySagaData : IContainSagaData{public virtual Guid Id { get; set; }public virtual string Originator { get; set; }public virtual string OriginalMessageId { get; set; }
[Unique]public virtual Guid MySagaId { get; set; }
}
ALTER TABLE [dbo].[MySagaData] ADD UNIQUE NONCLUSTERED ([MySagaId] ASC) ON [PRIMARY]
![Page 21: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/21.jpg)
Dennis van der Stelt
SAGA STATE
Memento
class MySagaData : IContainSagaData{public virtual Guid Id { get; set; }public virtual string Originator { get; set; }public virtual string OriginalMessageId { get; set; }
[Unique]public virtual Guid MySagaId { get; set; }
public virtual IList<Product> Products { get; set; }}
public class Product{public virtual Guid ProductId { get; set; }
}
![Page 22: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/22.jpg)
Dennis van der Stelt
SAGA STATE
Memento
class MySagaData : IContainSagaData{public virtual Guid Id { get; set; }public virtual string Originator { get; set; }public virtual string OriginalMessageId { get; set; }
[Unique]public virtual Guid MySagaId { get; set; }
public virtual IList<Product> Products { get; set; }}
public class Product{public virtual Guid ProductId MyUniqueId { get; set; }
}
![Page 23: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/23.jpg)
Dennis van der Stelt
Death to the batch jobBecause we don’t want to depend on operations ;-)
![Page 24: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/24.jpg)
Dennis van der Stelt
![Page 25: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/25.jpg)
Dennis van der Stelt
scheduled tasks
Your CEO had insomnia and was using the system in the middle of the night. The batch job failed somewhere in
the middle of updating 74 million records…
You need to figure out which row it failed on (how?), why it failed, correct the issue, then start the job again from where it left off, because if you have to start from the beginning, it won't get done before peak hours in
the morning.
because that sounds better than batch job
![Page 26: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/26.jpg)
Dennis van der Stelt
BATCH JOB
All customers that ordered $5000 in the last year, get preferred status
DateTime cutoff = DateTime.Today.AddDays(-365);
foreach(var customer in customers){var orderTotal = customer.Orders.Where(o => o.OrderDate > cutoff).Sum(order => order.OrderValue);
customer.Prefered = orderTotal > 5000;}
![Page 27: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/27.jpg)
Dennis van der Stelt
Tromsø, Norway
![Page 28: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/28.jpg)
Dennis van der Stelt
what if
we can see things before they happen?
![Page 29: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/29.jpg)
Dennis van der Stelt
customer preferred status
Dev: Let's say Steve orders something for $100. At what point does that amount no longer count toward Steve's preferred status?
BA: That's easy, after 365 days!
![Page 30: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/30.jpg)
Dennis van der Stelt
-$300-$100
+$300
DURABLE TIMEOUTS
Our way to predict the future
2015 2016
+$100
![Page 31: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/31.jpg)
Dennis van der Stelt
DURABLE TIMEOUTS
public void Handle(OrderPlaced message){
this.Data.CustomerId = message.CustomerId;
this.Data.RunningTotal += message.Amount;this.RequestTimeout<OrderExpired>(TimeSpan.FromDays(365),
timeout => timeout.Amount = message.Amount);
CheckForPreferredStatusChange();}
public void Handle(OrderExpired message){
this.Data.RunningTotal -= message.Amount;CheckForPreferredStatusChange();
}
![Page 32: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/32.jpg)
Dennis van der Stelt
DURABLE TIMEOUTS
private void CheckForPreferredStatusChange(){
if(this.Data.PreferredStatus == false && this.Data.RunningTotal >= 5000){
this.Bus.Publish<CustomerHasBecomePreferred>(evt => evt.CustomerId = this.Data.CustomerId);
}else if(this.Data.PreferredStatus == true && this.Data.RunningTotal < 5000){
this.Bus.Publish<CustomerHasBecomeNonPreferred(evt => evt.CustomerId = this.Data.CustomerId);
}}
![Page 33: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/33.jpg)
Dennis van der Stelt
Best PracticesThe silver bullets?
![Page 34: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/34.jpg)
Dennis van der Stelt
Rule #1 : Don’t query data
Never ever, ever, ever query data
- From the saga to another data source
- Owned by saga with a 3rd party tool
![Page 35: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/35.jpg)
race conditions do not exist
![Page 36: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/36.jpg)
Dennis van der Stelt
STARTING SAGAS
What business events can start a Saga?
class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>,IAmStartedByMessages<YourMessage>,IAmStartedByMessages<AnotherMesssage>
{public void Handle(MyMessage message){…
if (VerifyState())MarkAsComplete();
}}
![Page 37: Death of the batch job](https://reader031.vdocuments.site/reader031/viewer/2022021922/587b24311a28ab736c8b7383/html5/thumbnails/37.jpg)
sagas bring agility