![Page 1: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/1.jpg)
Andy Pavlo // Carnegie Mellon University // Spring 2016
ADVANCED DATABASE SYSTEMS
Lecture #22 – Vectorized Execution (Part II)
15-721
@Andy_Pavlo // Carnegie Mellon University // Spring 2017
![Page 2: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/2.jpg)
CMU 15-721 (Spring 2017)
CORRECTION
Original Xeon Phi (Knights Corner) uses in-order execution CPUs with a ring-based bus.
2016 Xeon Phi CPUs (Knights Landing) support out-of-order execution with tile architecture.
Three versions / form factors: → Self-boot Socket CPU → Self-boot Socket CPU + Fabric → PCI-E Device Co-Processor
2
Source: Avinash Sodani
![Page 3: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/3.jpg)
CMU 15-721 (Spring 2017)
CORRECTION
Original Xeon Phi (Knights Corner) uses in-order execution CPUs with a ring-based bus.
2016 Xeon Phi CPUs (Knights Landing) support out-of-order execution with tile architecture.
Three versions / form factors: → Self-boot Socket CPU → Self-boot Socket CPU + Fabric → PCI-E Device Co-Processor
2
Source: Avinash Sodani
![Page 4: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/4.jpg)
CMU 15-721 (Spring 2017)
TODAY’S AGENDA
Quickstep Bitweaving HyPer Data Blocks Project #3 Code Review Guidelines
3
![Page 5: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/5.jpg)
CMU 15-721 (Spring 2017)
OBSERVATION
The bit width of compressed data does not always fit naturally into SIMD register slots.
This means that the DBMS has to do extra work to transform data into the proper format.
4
![Page 6: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/6.jpg)
CMU 15-721 (Spring 2017)
BITWEAVING
Alternative storage layout for columnar databases that is designed for efficient predicate evaluation on compressed data using SIMD. → Order-preserving dictionary encoding. → Bit-level parallelization. → Only require common instructions (no scatter/gather)
Implemented in Wisconsin’s QuickStep engine. Became an Apache Incubator project in 2016.
5
BITWEAVING: FAST SCANS FOR MAIN MEMORY DATA PROCESSING SIGMOD 2013
![Page 7: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/7.jpg)
CMU 15-721 (Spring 2017)
BITWEAVING – STORAGE L AYOUTS
Approach #1: Horizontal
→ Row-oriented storage at the bit-level
Approach #2: Vertical
→ Column-oriented storage at the bit-level
6
![Page 8: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/8.jpg)
CMU 15-721 (Spring 2017)
HORIZONTAL STORAGE
7
0 0 1 1 0 1 1 1 0 0 0 1 1 1 0 1 0 0 0 0 0 1 1 1
t0
t1
t2
t3
t4
t5
t6
t7
1 0 0 0 1 1
t8
t9
Se
gm
en
t #
1
Se
gm
en
t #
2
=1
=5
=1
=6
=6
=4
=7
=0
=3
=4
![Page 9: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/9.jpg)
CMU 15-721 (Spring 2017)
Segment #2
HORIZONTAL STORAGE
7
0 0 1 1 0 1 1 1 0 0 0 1 1 1 0 1 0 0 0 0 0 1 1 1
t0
t1
t2
t3
t4
t5
t6
t7
1 0 0 0 1 1
t8
t9
Se
gm
en
t #
1
Se
gm
en
t #
2
Segment #1
v0 0 0 0 1 0 1 1 0
t0
t4
v1 0 1 0 1 0 1 0 0
t1
t5
v2 0 1 1 0 0 0 0 0
t2
t6
v3 0 0 0 1 0 1 1 1
t3
t7
v4 0 1 0 0 0 0 1 1
t8
t9
Processor Word
Processor Word
![Page 10: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/10.jpg)
CMU 15-721 (Spring 2017)
Segment #2
HORIZONTAL STORAGE
7
0 0 1 1 0 1 1 1 0 0 0 1 1 1 0 1 0 0 0 0 0 1 1 1
t0
t1
t2
t3
t4
t5
t6
t7
1 0 0 0 1 1
t8
t9
Se
gm
en
t #
1
Se
gm
en
t #
2
Segment #1
v0 0 0 0 1 0 1 1 0
t0
t4
v1 0 1 0 1 0 1 0 0
t1
t5
v2 0 1 1 0 0 0 0 0
t2
t6
v3 0 0 0 1 0 1 1 1
t3
t7
v4 0 1 0 0 0 0 1 1
t8
t9
Processor Word
Delimiter
Processor Word
![Page 11: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/11.jpg)
CMU 15-721 (Spring 2017)
BITWEAVING/H – EXAMPLE
8
SELECT * FROM table WHERE val < 5
t0
t4
0 0 0 1 0 1 1 0 X =
Y = 0 1 0 1 0 1 0 1
5
5
Source: Jignesh Patel
![Page 12: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/12.jpg)
CMU 15-721 (Spring 2017)
BITWEAVING/H – EXAMPLE
8
SELECT * FROM table WHERE val < 5
t0
t4
0 0 0 1 0 1 1 0 X =
Y = 0 1 0 1 0 1 0 1
5
5
mask = 0 1 1 1 0 1 1 1
(Y+(X⊕mask))∧¬mask= 1 0 0 0 0 0 0 0
Source: Jignesh Patel
![Page 13: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/13.jpg)
CMU 15-721 (Spring 2017)
BITWEAVING/H – EXAMPLE
Only requires three instructions to evaluate a single word.
Works on any word size and encoding length.
Paper contains algorithms for other operators.
8
SELECT * FROM table WHERE val < 5
t0
t4
0 0 0 1 0 1 1 0 X =
Y = 0 1 0 1 0 1 0 1
5
5
mask = 0 1 1 1 0 1 1 1
(Y+(X⊕mask))∧¬mask= 1 0 0 0 0 0 0 0
1 < 5
5 < 6
Source: Jignesh Patel
![Page 14: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/14.jpg)
CMU 15-721 (Spring 2017)
BITWEAVING/H – EXAMPLE
9
SELECT * FROM table WHERE val < 5
Source: Jignesh Patel
v0 0 0 0 1 0 1 1 0
t0
t4
v1 0 1 0 1 0 1 0 0
t1
t5
v2 0 1 1 0 0 0 0 0
t2
t6
v3 0 0 0 1 0 1 1 1
t3
t7
1 0 0 0 0 0 0 0
0 0 0 0 1 0 0 0
0 0 0 0 1 0 0 0
1 0 0 0 0 0 0 0 < 5
< 5
< 5
< 5
![Page 15: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/15.jpg)
CMU 15-721 (Spring 2017)
BITWEAVING/H – EXAMPLE
9
SELECT * FROM table WHERE val < 5
Source: Jignesh Patel
v0 0 0 0 1 0 1 1 0
t0
t4
v1 0 1 0 1 0 1 0 0
t1
t5
v2 0 1 1 0 0 0 0 0
t2
t6
v3 0 0 0 1 0 1 1 1
t3
t7
1 0 0 0 0 0 0 0
0 0 0 0 1 0 0 0
0 0 0 0 1 0 0 0
1 0 0 0 0 0 0 0 < 5
< 5
< 5
< 5 1 0 0 0 0 0 0 0
0 0 0 0 0 1 0 0
0 0 0 0 0 0 1 0
0 0 0 1 0 0 0 0 >>3
>>2
>>1
>>0
![Page 16: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/16.jpg)
CMU 15-721 (Spring 2017)
BITWEAVING/H – EXAMPLE
9
SELECT * FROM table WHERE val < 5
Source: Jignesh Patel
v0 0 0 0 1 0 1 1 0
t0
t4
v1 0 1 0 1 0 1 0 0
t1
t5
v2 0 1 1 0 0 0 0 0
t2
t6
v3 0 0 0 1 0 1 1 1
t3
t7
1 0 0 0 0 0 0 0
0 0 0 0 1 0 0 0
0 0 0 0 1 0 0 0
1 0 0 0 0 0 0 0 < 5
< 5
< 5
< 5 1 0 0 0 0 0 0 0
0 0 0 0 0 1 0 0
0 0 0 0 0 0 1 0
0 0 0 1 0 0 0 0 >>3
>>2
>>1
>>0
1 0 0 1 0 1 1 0 t
0 t
1 t
2 t
3 t
4 t
5 t
6 t
7
![Page 17: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/17.jpg)
CMU 15-721 (Spring 2017)
SELECTION VECTOR
SIMD comparison operators produce a bit mask that specifies which tuples satisfy a predicate. Have to convert it into offsets / positions. → Approach #1: Iteration → Approach #2: Pre-compute Positions Table
10
tuples = [ ] for (i=0; i<n; i++) { if sv[i] == 1 tuples.add(i); }
1 0 0 1 0 1 1 0 t
0 t
1 t
2 t
3 t
4 t
5 t
6 t
7
Selection Vector
![Page 18: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/18.jpg)
CMU 15-721 (Spring 2017)
SELECTION VECTOR
SIMD comparison operators produce a bit mask that specifies which tuples satisfy a predicate. Have to convert it into offsets / positions. → Approach #1: Iteration → Approach #2: Pre-compute Positions Table
10
1 0 0 1 0 1 1 0 t
0 t
1 t
2 t
3 t
4 t
5 t
6 t
7
Selection Vector
PAYLOAD KEY Positions Table
150
{0,3,5,6}
![Page 19: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/19.jpg)
CMU 15-721 (Spring 2017)
VERTICAL STORAGE
11
0 0 1 1 0 1 1 1 0 0 0 1 1 1 0 1 0 0 0 0 0 1 1 1
t0
t1
t2
t3
t4
t5
t6
t7
1 0 0 0 1 1
t8
t9
Se
gm
en
t #
1
Se
gm
en
t #
2
0 0 1 1 0 1 1 1 0 0 0 1 1 1 0 1 0 0 0 0 0 1 1 1
1 0 0 0 1 1
![Page 20: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/20.jpg)
CMU 15-721 (Spring 2017)
Segment #2
VERTICAL STORAGE
11
0 0 1 1 0 1 1 1 0 0 0 1 1 1 0 1 0 0 0 0 0 1 1 1
t0
t1
t2
t3
t4
t5
t6
t7
1 0 0 0 1 1
t8
t9
Se
gm
en
t #
1
Se
gm
en
t #
2
Segment #1
v0 0 1 1 0 1 1 0 1 t
0 t
1 t
2 t
3 t
4 t
5 t
6 t
7
v1 0 0 1 0 1 0 0 1
v2 1 1 0 1 0 0 0 1
Processor Word
v3 1 0 0 0 0 0 0 0 t
8 t
9 -
-
-
-
-
-
v4 0 1 0 0 0 0 0 0
v5 0 1 0 0 0 0 0 0
![Page 21: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/21.jpg)
CMU 15-721 (Spring 2017)
BITWEAVING/V – EXAMPLE
12
Segment #1
v0 0 1 1 0 1 1 0 1 t
0 t
1 t
2 t
3 t
4 t
5 t
6 t
7
v1 0 0 1 0 1 0 0 1
v2 1 1 0 1 0 0 0 1
SELECT * FROM table WHERE key = 2
0 1 0
![Page 22: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/22.jpg)
CMU 15-721 (Spring 2017)
BITWEAVING/V – EXAMPLE
12
Segment #1
v0 0 1 1 0 1 1 0 1 t
0 t
1 t
2 t
3 t
4 t
5 t
6 t
7
v1 0 0 1 0 1 0 0 1
v2 1 1 0 1 0 0 0 1
SELECT * FROM table WHERE key = 2
0 1 0
1 0 0 1 0 0 1 0
0 0 0 0 0 0 0 0
SIMD Compare
![Page 23: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/23.jpg)
CMU 15-721 (Spring 2017)
BITWEAVING/V – EXAMPLE
12
Segment #1
v0 0 1 1 0 1 1 0 1 t
0 t
1 t
2 t
3 t
4 t
5 t
6 t
7
v1 0 0 1 0 1 0 0 1
v2 1 1 0 1 0 0 0 1
SELECT * FROM table WHERE key = 2
0 1 0
1 0 0 1 0 0 1 0
0 0 0 0 0 0 0 0
SIMD Compare
1 1 1 1 1 1 1 1
SIMD Compare
![Page 24: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/24.jpg)
CMU 15-721 (Spring 2017)
BITWEAVING/V – EXAMPLE
12
Segment #1
v0 0 1 1 0 1 1 0 1 t
0 t
1 t
2 t
3 t
4 t
5 t
6 t
7
v1 0 0 1 0 1 0 0 1
v2 1 1 0 1 0 0 0 1
SELECT * FROM table WHERE key = 2
0 1 0
1 0 0 1 0 0 1 0
0 0 0 0 0 0 0 0
Can perform early pruning just like in BitMap indexes.
The last vector is skipped because all bits in previous comparison are zero. SIMD Compare
1 1 1 1 1 1 1 1
SIMD Compare
![Page 25: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/25.jpg)
CMU 15-721 (Spring 2017)
EVALUATION
Single-threaded execution of a single query derived from TPC-H benchmark. → Selectivity: 10%
10GB TPC-H Database → 1 billion tuples → Uniform distribution
13
Source: Jignesh Patel
SELECT COUNT(*) FROM R WHERE R.a < C
![Page 26: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/26.jpg)
CMU 15-721 (Spring 2017)
EVALUATION
14
0
2
4
6
8
10
1 11 21 31
Cy
cle
s /
C
od
e
Size of Code (# of Bits)
Naïve SIMD Scan BitWeaving/V BitWeaving/H
TPC-H Aggregation Query
Intel Xeon X5650 @ 2.66 GHz
Source: Jignesh Patel
SIMD Parallelization
Fewer Cache Misses
Early Pruning
![Page 27: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/27.jpg)
CMU 15-721 (Spring 2017)
OBSERVATION
At a high-level, BitWeaving is essentially a bitmap index. This is great for OLAP, bad for OLTP.
How can compress data in such a way that still allow for efficient point queries?
15
![Page 28: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/28.jpg)
CMU 15-721 (Spring 2017)
HYPER – DATA BLOCKS
Individually determine the best suitable compression scheme per block. → Hot tuples are stored uncompressed (OLTP) → Cold tuples are stored in compressed Data Blocks (OLAP)
Each Data Block is compressed using the best scheme for its tuples.
16
DATA BLOCKS: HYBRID OLTP AND OLAP ON COMPRESSED STORAGE USING BOTH VECTORIZATION AND COMPILATION SIGMOD 2016
![Page 29: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/29.jpg)
CMU 15-721 (Spring 2017)
OBSERVATION
Multiple storage layouts and compression schemes mean the DBMS has to generate multiple code paths per query plan.
This increases the compilation time per query and the total size of the query plan cache.
17
![Page 30: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/30.jpg)
CMU 15-721 (Spring 2017)
HYPER – SPLIT EXECUTION MODELS
Compressed Data: Vectorized Execution
→ Use the pre-compiled primitives from Vectorwise.
Uncompressed Data: LLVM Compilation
→ Data is always stored in the same format so there is only ever one code path to compile.
18
![Page 31: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/31.jpg)
CMU 15-721 (Spring 2017)
PARTING THOUGHTS
Just like in query compilation, getting the best performance with vectorization requires the DBMS to store data in a way that is best for the CPU and not the best for humans’ understanding.
19
![Page 32: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/32.jpg)
CMU 15-721 (Spring 2017)
ADMINISTRIVIA
Code Review #1 Submission: April 11th
Project Status Meetings: April 13th
Project Status Updates: April 18th
Code Review #1 Completion: April 18th
20
![Page 33: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/33.jpg)
CMU 15-721 (Spring 2017)
CODE REVIEWS
Each group will send a pull request to the CMU-DB master branch. → This will automatically run tests + coverage calculation. → PR must be able to merge cleanly into master branch. → Reviewing group will write comments on that request. → Add the URL to the Google spreadsheet and notify the
reviewing team that it is ready.
Please be helpful and courteous.
21
![Page 34: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/34.jpg)
CMU 15-721 (Spring 2017)
GENERAL TIPS
The dev team should provide you with a summary of what files/functions the reviewing team should look at.
Review fewer than 400 lines of code at a time and only for at most 60 minutes.
Use a checklist to outline what kind of problems you are looking for.
22
Source: SmartBear
![Page 35: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/35.jpg)
CMU 15-721 (Spring 2017)
CHECKLIST – GENERAL
Does the code work?
Is all the code easily understood?
Is there any redundant or duplicate code?
Is the code as modular as possible?
Can any global variables be replaced?
Is there any commented out code?
Is it using proper debug log functions?
23
Source: Gareth Wilson
![Page 36: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/36.jpg)
CMU 15-721 (Spring 2017)
CHECKLIST – DOCUMENTATION
Do comments describe the intent of the code?
Are all functions commented?
Is any unusual behavior described?
Is the use of 3rd-party libraries documented?
Is there any incomplete code?
24
Source: Gareth Wilson
![Page 37: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/37.jpg)
CMU 15-721 (Spring 2017)
CHECKLIST – TESTING
Do tests exist and are they comprehensive?
Are the tests actually testing the feature?
Are they relying on hardcoded answers?
What is the code coverage?
25
Source: Gareth Wilson
![Page 38: CMU SCS 15-721 (Spring 2017) :: Vectorized Execution (Part II) · CMU 15-721 (Spring 2017) BITWEAVING . Alternative storage layout for columnar databases that is designed for efficient](https://reader035.vdocuments.site/reader035/viewer/2022063001/5f165d4589f53879ac209bca/html5/thumbnails/38.jpg)
CMU 15-721 (Spring 2017)
PROJECT #3 – STATUS UPDATE
I will meet with groups individually on Thursday.
Next Tuesday each group will give a five minute presentation to update the class about the current status of your project.
Each presentation should include: → Current development status. → Whether anything in your plan has changed. → Any thing that surprised you.
26