what reika taught us
TRANSCRIPT
What Reika Taught usLessons learned from building Reika, a DSL for TSDB
Pinglei Guo & Chujiao Hou
Agenda● What is Reika● What is Time Series and Time Series Database (03/21 if interested CMPS 278)
● Design Goal● Staging achievements● Implementation detail● Lessons learned● Acknowledgement
What is Reika
- General language + Query language- Built in function for running SQL like query on various Time Series Databases- Depend on TSDB-Proxy for execution
Design Goal- SQL like syntax- Support various backend
Staging achievement- Only the general programming language part is implemented- written in Java (1500 + 1000 (generated by ANTLR) )- REPL- Editor support (vscode)
REPL (Read Eval Print Loop)
VS Code Editor Plugin
Implementation Detail- Lexer and Parser using ANTLR4 (ANother Tool for Language Recognition) - Build AST from parse tree with symbol and type checking - Static typing- Interpreter & Shell in Java
ANTLR Grammar
prog : stat+ ;
type : 'int' | 'double' | 'bool' | 'string' | 'date';
varDeclare : type ID '=' expr ';';
varAssign : ID '=' expr ';';
stat : varDeclare
| varAssign
| expr ';'
;
● http://www.antlr.org/● From a grammar, ANTLR generates a parser
that can build and walk parse trees
Grammar: Program & Statements
ANTLR Grammar
expr : INT
| DOUBLE
| BOOL
| STRING
| ID '(' exprList? ')'
| ID
| expr MULT expr
| expr DIV expr
| expr ADD expr
| expr MINUS expr
| expr AND expr
| expr OR expr
;
exprList : expr (',' expr)* ;
Grammar: expression & expression list
ANTLR Grammar
INT : [0-9]+ ;
DOUBLE: [0-9]+ '.' [0-9]+ ;
BOOL : 'true' | 'false' ;
STRING: '"' (ESC|.)*? '"';
ESC : '\\"' | '\\\\' ;
ID : LETTER (LETTER | [0-9])* ;
LETTER : [a-zA-Z] ;
ADD : '+';
MINUS: '-';
MULT: '*';
DIV: '/';
AND : '&&';
OR : '||' ;
Grammar: Token
WS : [ \t\n\r]+ -> channel(HIDDEN) ;
COMMENT : '/*' .*? '*/' -> channel(HIDDEN) ;
SL_COMMENT : '//' .*? '\n' -> channel(HIDDEN);
ANTLR Grammar
Grammar: Token (ignored)
Symbol Checking
● Undefined variable
x = 1;
● Duplicate definition
int x = 1;
double x = 2.0;
● 1 is int -> x is int (type inference)
int x = 1;
● Replace the old one with new one in
symbol table
double x = 2.0;
Error Recovery
Recovery != Recovered AST node should be passed to next step ( interpreter etc.)
Type Checking
Error Recovery
● Incompatible assign
string chinaMobile = 10086;
● incompatible type
"elder" + 1;
● unsupported operation
"elder" - "1s";
● Use lhs type
string chinaMobile = “”
● Use Any Type
● Use Any Type
Symbol & Type checking example
You often don't really understand the problem until after the first time you implement a solution
- The Cathedral and the Bazaar
Lessons
Lessons
Check the library's source before you use it
When shit things happen, they happen recursively
ANTLR Golang runtime
It’s Run Time
The original Reika repository, written in Golang
Related work● InfluxQL & Tick Script● HeroicQL● Juttle ● SparkQL● LINQ
Acknowledgement● Zheyuan Chen & Prof. Alvaro in CMPS 232
Thank You!
Source code:
https://github.com/xephonhq/tsdb-proxy-java
Authors:
Pinglei Guo [email protected] Hou [email protected]