Building a highly scalable exchangeTo react or not to react

Sreekanth G SChief Technology OfficerHatio Innovations Private LimitedA BillDesk Company

Me

Thinker Blogger

Technologist

Programmer

Traveler

Writer

Crypto Evangelist

Ideologist

2

What this presentation is not

A coding exerciseA programming tutorialA technology recommendationOr a demo presentation

</>3

Instead it is about

Our self-learningAnd a retrospection on

What we didAnd what we did wrong

4

We built a crypto currency exchange

5

Our initial stack

Ruby on RailsPostgreSQLDockerised DeploymentsOrchestration using Rancher

6

The best part

Very easy to buildShortest time to marketHandles things pretty wellUntil…

7

The woes

Scaling Experience

Availability

8

What went wrong

The stack couldn’t scale

Rails view rendering wasn’t efficient

Nothing but jQuery in Frontend

And Rails for Backend wasn’t the solution

9

RCA – The three letter horror

Heavy long running SQL queries

Lack of efficient caching

502 Bad Gateways on Upstream unavailability

No way to let customers know that we are down

10

Data, data, data

Almost a single page app.Most time spent on Trading.

Data is extremely important. Realtime updates.Customers make decisions based on data.

11

Realtime updates

WebSocket updates

Charts & Graphs

XHR Requests

Events, alerts and notifications

12

Scaling – The mammoth task

Scaling is a multistep solutionSplit into - Fixing the SQL queries - Caching of responses- Decoupling the frontend

13

Fixing the backend issues

SQL Query Optimizations – Moving to Aurora

Caching of requests & data – Memcached & Redis

Archiving of stale data – Adopting the ELK Stack

Taking care of complex weightlifting – C# & .NET

14

Initial JavaScript days

For simple DOM manipulations

Handling of event life cycles – WebSocket updates

using jQuery bundled with Rails

and Vue.js for simple data bindings

15

Limitations of this setup

Cluttered and spaghetti JavaScript code

Never ending conflicts across multiple framework

Frontend was still tightly coupled with backend

View rendering was a toll on App Server

16

Decoupling the frontend

Deciding on a single frontend framework

Multiple contenders –

Vue, React, Backbone and Angular

Roadmap for Mobile Apps – Android & iPhone

17

The difficult choice

18

Vue React

Lightweight Flexible

Vue Specific Components Pure JavaScript

Better Documentation Larger Community

HTML Templating Optional JSX Templating

Very easy to Integrate Higher learning curve

Limited support for Mobile Apps React Native

Difficult to hire engineers Very good talent pool

19

The shortlist

React it is.

Rich Library Support

Very good talent pool

React Native for Apps

20

The learning curve

Too minimalistic

Concept of a state store was new

One-way data binding was challenging

Not so good documentation

21

JSX as a pain point

Developers used to write HTML/ErbDifficulty and inertia learning JSX

22

The good part - Components

Code became components

- Reusable code, with Single Responsibility

- Easier to manage

- Without any or very low side effects

23

Components & Actionslet handleMarketTradeHistory = activeMarket => { return dispatch => { const marketTradeHistoryChannel = pusher.subscribe( `market-${activeMarket}-trade-history` ); marketTradeHistoryChannel.bind("market_trade_history", data => { data.trade_history.map(item => dispatch(processTradeHistoryData(item))); }); }; };

24

Dispatchers to do the workexport function processTradeHistoryData(data) { return dispatch => { dispatch({ type: ActionType.ADD_TRADE_ORDER, payload: { amount: data.a, created_at: data.T, order_type: data.M, price: data.p, vol: data.q } }); }; }

25

The good part – Clean Data

Data is better handled

- Single storage of data and state

- One-way data binding, with clean hierarchy

- You cannot accidentally pollute data

26

The good part – Invisible DOM

The declarative approach

- You do not directly touch DOM

- Updates are batched and automatic

- Think about what and not how

27

100s of lines of code to a few

Componentization made code smaller

Legacy JavaScript code wasn’t reusable

No way to debug or test

Now it’s decoupled and self handling

28

MVC to Pure API

Functionality interwoven with View

Single codebase, hard to maintain

Transition to Pure API, everything returns data

View can be anything, Web or Mobile!

29

React Native

Future Apps in React Native

Pure API model complements this approach

Frontend developers can collaborate

UX and Apps coming under an umbrella

30

BenchmarkingLoad times between page loads and data population:

Category Load Time

RoR with No Optimization 2.80s

RoR with Backend Optimization 2.03s

RoR with React Frontend 1.80s

RoR with Backend Optimization & React Frontend 1.07s

31

Unsolved Issues

Virtual DOM is at times slow

Upgrading and updating is not easy as expected

High memory consumption – 16.3 MB

Slow startup times – 219 ms

32

The learnings

Scale is not a frontend/backend only problem

Customers tolerate informed issues, not 502s

Developer happiness is extremely important

Initial learning curve can be future productivity

33

The takeaways

Rewriting entire code base isn’t easy.

When you are forced to do it, decide wisely.

React is not a one stop solution, it was a fit for us.

At the end of the day, its about delivering.

34

Thank You@sreekanthgs

35