![Page 1: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/1.jpg)
CMU 15-445/645 (Fall 2017)
Index Suggestion InPeloton
Priyatham BollimpalliSivaprasad Sudhir
Vamshi Reddy Konagari
![Page 2: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/2.jpg)
CMU 15-445/645 (Fall 2017)
Infrastructure
● We have access to the query history table, stats, etc. at the brain end which can be used by the optimizer
Catalog Catalog
SQL Engine
Catalog Sync
ServerBrain
Tables
...
![Page 3: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/3.jpg)
CMU 15-445/645 (Fall 2017)
What’s in the brain??
● Brain job - scheduled every △ seconds.
● Index selection Job● Algorithm bases its cost
on the optimizer cost model
● Suggested configuration installed through RPC
Brain
Index Selection Add/Drop Index RPC
Brain Job(Recurring)
Catalogs
Optimizer
An Efficient, Cost-Driven Index Selection Tool for Microsoft SQL Server Surajit Chaudhuri, Vivek Narasayya VLDB Very Large Data Bases Endowment Inc. August 1, 1997
![Page 4: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/4.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection Algorithm
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
{ {a}, {b, a}, {b, c} }
#indexes?#columns in each index?
Parameters?
Best Indexes?
![Page 5: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/5.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection Algorithm
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
{ {a}, {b, a}, {b, c} }
#indexes k = 2#columns in each index n = 3
Parameters?
Best Indexes?
![Page 6: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/6.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Some candidates to start with
![Page 7: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/7.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Some candidates to start with
![Page 8: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/8.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Some candidates to start with
{ {a}, {b}, {c}, {d}, {e}, {f} }
![Page 9: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/9.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3; {
{a}, {b}, {c}, {d}, {e}, {f} }
Admissible Index
Generation
![Page 10: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/10.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
CI = { {a}, {b}, {c}, {d}, {e}, {f} }
Need to choose k indexes from these
![Page 11: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/11.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
CI = { {a}, {b}, {c}, {d}, {e}, {f} }
Need to choose k indexes from theseNaively enumerate and choose the best one from all subsets of size < k?
![Page 12: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/12.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
CI = { {a}, {b}, {c}, {d}, {e}, {f} }
Need to choose k indexes from theseNaively enumerate and choose the best one from all subsets of size < k?
#indexes?#columns in each index?#naive enumeration threshold?
![Page 13: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/13.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
CI = { {a}, {b}, {c}, {d}, {e}, {f} }
Need to choose k indexes from theseNaively enumerate and choose the best one from all subsets of size < k?
#indexes k = 2#columns in each index m = 3#naive enumeration threshold n = 1
![Page 14: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/14.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
CI = { {a}, {b}, {c}, {d}, {e}, {f} }
Naively enumerate all subsets of size < n and choose the best one from it
![Page 15: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/15.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
CI = { {a}, {b}, {c}, {d}, {e}, {f} }
Naively enumerate all subsets of size < n and choose the best one from it
CI = { {}, {a}, {b}, {c}, {d}, {e}, {f} }
![Page 16: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/16.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
CI = { {a}, {b}, {c}, {d}, {e}, {f} }
We need 2 indexesCI = { {b} }
![Page 17: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/17.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
CI = { {a}, {b}, {c}, {d}, {e}, {f} }
Greedily choose the next best
CI = { {b} }
![Page 18: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/18.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
CI = { {a}, {b}, {c}, {d}, {e}, {f} }
Greedily choose the next best
CI = { {b} }
![Page 19: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/19.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
CI = { {a}, {b}, {c}, {d}, {e}, {f} }
Greedily choose the next best
CI = { {b},{a} } { {b},{c} } { {b},{d} } { {b},{e} } { {b},{f} }
![Page 20: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/20.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
CI = { {a}, {b}, {c}, {d}, {e}, {f} }
Greedily choose the next best
CI = { {b},{a} } { {b},{c} } { {b},{d} } { {b},{e} } { {b},{f} }
![Page 21: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/21.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
CI = { {b}, {d} }
Configuration Enumeration
![Page 22: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/22.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
CI = { {b}, {d} }
Multi-column indexes?
Configuration Enumeration
![Page 23: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/23.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
CI = { {b}, {d} }
Multi-column indexes?
Configuration Enumeration
CI = CI ∪ (CI ✕ AI)
![Page 24: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/24.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
CI = { {b}, {d} }
Configuration Enumeration
CI = CI ∪ (CI ✕ AI)CI = {
{b}, {d}, {ba}, {bc}, {bd}, {be}, {bf},{da}, {db}, {dc}, {de}, {df},
}
![Page 25: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/25.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
Configuration Enumeration
CI = CI ∪ (CI ✕ AI)CI = {
{b}, {d},{ba}, {bc}, {bd}, {be}, {bf},{da}, {db}, {dc}, {de}, {df},
}
Multi-Column Index Generation
![Page 26: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/26.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
Configuration Enumeration
CI = { {b}, {d},{ba}, {bc}, {bd}, {be}, {bf},{da}, {db}, {dc}, {de}, {df},
}
Multi-Column Index Generation
![Page 27: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/27.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
Configuration Enumeration
CI = { {b}, {d},{ba}, {bc}, {bd}, {be}, {bf},{da}, {db}, {dc}, {de}, {df},
}
Multi-Column Index Generation
![Page 28: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/28.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
Configuration Enumeration
CI = { {b}, {d},{ba}, {bc}, {bd}, {be}, {bf},{da}, {db}, {dc}, {de}, {df},
}
Multi-Column Index Generation
Candidate Selection
![Page 29: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/29.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
Configuration Enumeration
CI = { {b}, {d}, {ba}, {bc}, {dc}, {de}, {df} }
Multi-Column Index Generation
Candidate Selection
![Page 30: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/30.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
Configuration Enumeration
CI = { {b}, {d}, {ba}, {bc}, {dc}, {de}, {df} }
Multi-Column Index Generation
Candidate Selection
![Page 31: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/31.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE a = 1;SELECT * FROM FOO WHERE a = 1;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
Configuration Enumeration
...Multi-Column Index
Generation
Candidate Selection
![Page 32: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/32.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
Configuration Enumeration
CI = { {bca}, {def} }
Multi-Column Index Generation
Candidate Selection
Best k indexes
![Page 33: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/33.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
Configuration Enumeration
Multi-Column Index Generation
Candidate Selection
![Page 34: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/34.jpg)
CMU 15-445/645 (Fall 2017)
Index Selection AlgorithmSELECT * FROM FOO WHERE b = 1;SELECT * FROM FOO WHERE d = 7;
AI = { {a}, {b}, {c}, {d}, {e}, {f} }
SELECT * FROM FOO WHERE b = 7;SELECT * FROM FOO WHERE b = 9 and c = 10;SELECT * FROM FOO WHERE a = 1 and b = 2 and c = 3;SELECT * FROM FOO WHERE c = 1 and b = 1 and a = 3;SELECT * FROM FOO WHERE d = 7;SELECT * FROM FOO WHERE d = 9 and e = 10;SELECT * FROM FOO WHERE d = 1 and e = 2 and f = 3;
Admissible Index Generation
Configuration Enumeration
Multi-Column Index Generation
Candidate Selection
What-If API
![Page 35: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/35.jpg)
CMU 15-445/645 (Fall 2017)
Testing Framework
● We wrote extensive tests to ensure the correctness of the index selection algorithm and what if index API.
● Found many bugs cost model/optimizer.● Wrote a generic framework for creating workloads useful
for Brain Index Suggestion algorithms.● Then we thought of running the benchmarks...
![Page 36: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/36.jpg)
CMU 15-445/645 (Fall 2017)
Fixes, fixes and fixes...● TPCC - throughput 9 queries/sec with or without indexes● Statement cache not flushed on index creation● Index creation takes ages (on string, floats columns)● Analyze stats is not used by anyone - crashes on using it
for the second time concurrently.● Optimizer never considered multi-column indexes● Optimizer - explicitly have to call Analyze stats - which
segfaults if you call twice● TPCC without primary keys segfaults.● Prepared statements were not being logged
![Page 37: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/37.jpg)
CMU 15-445/645 (Fall 2017)
TPC-C Throughput
Workload: TPCC
Without Indexes: No primary or secondary indexes
With DBA Indexes: Primary and secondary indexes in the workload
With Brain Indexes: Start with no indexes and use the indexes suggested by our algorithm
![Page 38: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/38.jpg)
CMU 15-445/645 (Fall 2017)
Number of tuples accessed
![Page 40: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/40.jpg)
CMU 15-445/645 (Fall 2017)
Algorithm Runtime - Number of queries
#indexes = 10#columns = 2#naive enumeration threshold = 2
![Page 41: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/41.jpg)
CMU 15-445/645 (Fall 2017)
Algorithm Runtime - Naive enumeration threshold
#indexes = 10#columns = 3#queries = 2000
![Page 42: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/42.jpg)
CMU 15-445/645 (Fall 2017)
Algorithm Runtime - Number of indexes chosen
#columns= 3#naive enumeration threshold = 2#queries = 1000
![Page 43: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/43.jpg)
CMU 15-445/645 (Fall 2017)
Algorithm Runtime - Number of columns / iterations
#indexes = 10#naive enumeration threshold = 2#queries = 1000
![Page 44: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/44.jpg)
CMU 15-445/645 (Fall 2017)
Future Work
● Tuning the knobs● Incorporate memory usage of the indexes created● Online version
![Page 45: Index Suggestion In Vamshi Reddy Konagari Priyatham ... · Priyatham Bollimpalli Sivaprasad Sudhir Vamshi Reddy Konagari. CMU 15-445/645 (Fall 2017) Infrastructure We have access](https://reader030.vdocuments.site/reader030/viewer/2022041016/5ec7e69b9b761d7a4112ab37/html5/thumbnails/45.jpg)
CMU 15-445/645 (Fall 2017)
Thanks :)
Questions?