building a scalable system for shipping tracking

BUILDING A SCALABLE SYSTEM FOR SHIPPING TRACKING

PREMSHREE PILLAI,

Sunday, October 5, 14

SR. ENGINEER, SHIPPINGPREVIOUSLY, YAHOO!

@premshree


https://www.etsy.com/listing/110699162/seeker-of-happiness-canvas-tote-bag-you

RE-IMAGINE COMMERCEIN WAYS THAT BUILD A MORE FULFILLING AND LASTING WORLD




CLEVELAND, OHIO

https://www.etsy.com/listing/199433520/1893-antique-map-of-cleveland-ohio


HONG KONG

https://www.etsy.com/listing/199835452/hong-kong-china-morning-street-view


MALI PLAC NA TAVANU, ZAGREB


♥ POTTERY


SCOTTSDALE, AZ


ANDOVER, MA


LEFKIS, GREECE


https://www.etsy.com/listing/150742570/white-nesting-bowl-set-modern-ceramic

1 MILLION SELLERS40 MILLION MEMBERS26 MILLION ACTIVE LISTINGS




Where is myPACKAGE?

https://www.flickr.com/photos/oneeighteen/2597603677


Where is myPACKAGE?

https://www.flickr.com/photos/oneeighteen/2597603677

♥


IN TRANSIT 200,000+

https://www.etsy.com/listing/153268931/custom-world-map-absract-print-poster


OUT FOR DELIVERY 125,000+



SHIPPING CARRIERS 150+



SHIPPING CARRIERS


FedEx

DHL

USPS

Hrvatska pošta

...


LET’S DIVE IN


tracking_events


SHIPPING APIS


GET http://production.shippingapis.com/ShippingAPI.dll?API=TrackV2&XML=%3CTrackFieldRequest+USERID%3D%22022XXXXH6696%22%3E%3CTrackID+ID%3D%22LZ13XXX3064US%22%3E%3C%2FTrackID%3E%3C%2FTrackFieldRequest%3E


TRACKING STATES


INFO_RECEIVED

PACKAGE_RECEIVED

IN_TRANSIT

DELAYED

OUT_FOR_DELIVERY

...

DELIVERED


STATE MAP


INFO_RECEIVED

“acceptance”

“processed at usps origin facility"

“accepted at usps origin sort facility”

“picked up”


INFO_RECEIVED

“tendered at fedex office”

“picked up"

“in fedex possession”


IN_TRANSIT

“dispatched from usps sort facility”

“sorting complete"

“local holiday - will attempt delivery on next working day”

“customs clearance processing


SELECT * FROM trackings WHERE last_access_date < ?


? = now() - 15


WHAT’S THE PROBLEM?


SIZE


SELECT * FROM trackings WHERE last_access_date < ? LIMIT ?


STRAGGLERS


SHARDING 101


SELECT * FROM trackings WHERE shard = ? AND last_access_date < ?


QUEUE


FAUX QUEUE


QUEUE_3_DAY

QUEUE_1_DAY

QUEUE_3_HOUR

QUEUE_1_HOUR

...

QUEUE_STOPPED


QUEUE MAP


INFO_RECEIVED => QUEUE_1_DAY

PACKAGE_RECEIVED => QUEUE_1_DAY

IN_TRANSIT => QUEUE_1_DAY

OUT_FOR_DELIVERY => QUEUE_1_HOUR

...

DELIVERED => QUEUE_STOPPED


INFO_RECEIVED => QUEUE_3_HOUR

PACKAGE_RECEIVED => QUEUE_3_HOUR

IN_TRANSIT => QUEUE_3_HOUR

OUT_FOR_DELIVERY => QUEUE_1_HOUR

...

DELIVERED => QUEUE_STOPPED


QUEUE AGE


QUEUE_3_DAY => 1*60*60*24*3

QUEUE_1_DAY => 1*60*60*24

QUEUE_3_HOUR => 3*60*60

QUEUE_1_HOUR => 1*60*60


SELECT * FROM trackings WHERE shard = ? AND queue = ? AND last_access_date < ?


last_access_date < time() - age


ASYNC


IN SUMMARY


API

STATE MAP

SHARDING

CRON/ASYNC

FAUX-QUEUE


https://www.etsy.com/listing/200465416/mustache-card-youre-awesome


Hvala!https://www.etsy.com/listing/157215176/croatia-watercolor-painting-art-print


https://www.etsy.com/listing/192250796/i-mustache-you-a-question-embroidery-4


building a scalable system for shipping tracking

Technology

queue mapsunday

queue agesunday

trackingswhere shard

shipping carrierssunday

shipping apissunday

tracking statessunday

state mapsunday

faux queuesunday