sns data flow
DESCRIPTION
TRANSCRIPT
![Page 1: SNS Data Flow](https://reader030.vdocuments.site/reader030/viewer/2022020105/54c454534a7959e4118b462d/html5/thumbnails/1.jpg)
SNS DataFlow
- social network service -
gmind7
![Page 2: SNS Data Flow](https://reader030.vdocuments.site/reader030/viewer/2022020105/54c454534a7959e4118b462d/html5/thumbnails/2.jpg)
Architecture (DB)
- Fotolog(2007)
![Page 3: SNS Data Flow](https://reader030.vdocuments.site/reader030/viewer/2022020105/54c454534a7959e4118b462d/html5/thumbnails/3.jpg)
- Fotolog(2007)
Architecture (DB)
![Page 4: SNS Data Flow](https://reader030.vdocuments.site/reader030/viewer/2022020105/54c454534a7959e4118b462d/html5/thumbnails/4.jpg)
- Fotolog(2007)
Architecture (DB)
![Page 5: SNS Data Flow](https://reader030.vdocuments.site/reader030/viewer/2022020105/54c454534a7959e4118b462d/html5/thumbnails/5.jpg)
- Fotolog(2007)
Architecture (DB)
![Page 6: SNS Data Flow](https://reader030.vdocuments.site/reader030/viewer/2022020105/54c454534a7959e4118b462d/html5/thumbnails/6.jpg)
- Flickr (Yahoo) 2005
Architecture (DB)
![Page 7: SNS Data Flow](https://reader030.vdocuments.site/reader030/viewer/2022020105/54c454534a7959e4118b462d/html5/thumbnails/7.jpg)
- Facebook, twitter, Flickr……
Architecture (Cache)
worst best
memcached
![Page 8: SNS Data Flow](https://reader030.vdocuments.site/reader030/viewer/2022020105/54c454534a7959e4118b462d/html5/thumbnails/8.jpg)
- MySQL는 사용하고 있지만 RDB로서가 아니다.
Architecture (Cache)
memcached
![Page 9: SNS Data Flow](https://reader030.vdocuments.site/reader030/viewer/2022020105/54c454534a7959e4118b462d/html5/thumbnails/9.jpg)
- 대량 메세지 통신은 비동기 (난 편지를 우체국에 전달해 주면 끝)
- ActiveMQ 에서 RabbitMQ로 전환
Architecture (queue)
![Page 10: SNS Data Flow](https://reader030.vdocuments.site/reader030/viewer/2022020105/54c454534a7959e4118b462d/html5/thumbnails/10.jpg)
timeline_1 timeline_2 timeline_3 timeline_4
Index DB
엑스맨
글인덱스 글인덱스 글인덱스 글인덱스
1. Index DB (master+1, slave+3)
- 테이블 명 : timeline_#tableIndex#
- 테이블 수 : 36개 (변동가능 : 현재 목표 3%이하)
2. 데이터 DB (master+1, slave+3)
- 테이블 명 : post_txt
Table 정보
post_txt
원본글
Data DB
아이언맨
거북이
오토바이
슈퍼맨
바나나
사과
기차
Architecture (MySQL)
![Page 11: SNS Data Flow](https://reader030.vdocuments.site/reader030/viewer/2022020105/54c454534a7959e4118b462d/html5/thumbnails/11.jpg)
TimeLine Index HashingKey
![Page 12: SNS Data Flow](https://reader030.vdocuments.site/reader030/viewer/2022020105/54c454534a7959e4118b462d/html5/thumbnails/12.jpg)
timeline_1 timeline_2 timeline_3 timeline_4
timeline_0
RealTime BackGround
friend1 friend2 friend3 friend4
post_txt
1. 인덱스 정보를 가져오기 위한 쿼리
SELECT post_txt_seq
FROM timeline_#tableIndex#
WHERE playncAcctId=#userKey#
LIMIT #startRow#, #fetchSize#
※ post_txt_seq 리스트가 IN 쿼리의 파라미터로 사용된다.
2. 게시물의 정보를 가져오기 위한 쿼리
SELECT *
FROM post_txt
WHERE post_txt_seq in (#post_txt_seq#, (#post_txt_seq#,…)
post
원본글 글인덱스
글인덱스 글인덱스 글인덱스 글인덱스
PostFlow
Memcached Server
Query 가져오기
![Page 13: SNS Data Flow](https://reader030.vdocuments.site/reader030/viewer/2022020105/54c454534a7959e4118b462d/html5/thumbnails/13.jpg)
PostFlow FrontEnd BackEnd
SNS WAS
Ingame WAS
RabbitMQ Server MessageConsumer SNS Index DB Server
1. Spring Framework Task (50개의 Thread 병렬 처리)
2. 각 Thread 는 420개씩 Batch Insert 처리
![Page 14: SNS Data Flow](https://reader030.vdocuments.site/reader030/viewer/2022020105/54c454534a7959e4118b462d/html5/thumbnails/14.jpg)
Index DB Extension
timeline_1 timeline_2 timeline_3 timeline_4
글인덱스 글인덱스 글인덱스 글인덱스
SNS Index DB Server_1 (4TB)
max
timeline_1
글인덱스
SNS Index DB Server_2 (4TB)
슈퍼맨 배트맨 아이언맨 엑스맨 슈퍼맨
multiDataSource Connect
![Page 15: SNS Data Flow](https://reader030.vdocuments.site/reader030/viewer/2022020105/54c454534a7959e4118b462d/html5/thumbnails/15.jpg)
Data DB Extension
SNS Data DB Server_1 (4TB)
post_txt
SNS Data DB Server_2 (4TB)
프로필
프로필
그룹
프로필
방명록
방명록
max
그룹
그룹
그룹
그룹
post_txt