libattachsql, the next-generation c connector for mysql
DESCRIPTION
Slides for my libAttachSQL talk for Percona Live London 2014TRANSCRIPT
libAttachSQLThe Next-Generation C Connector for MySQL
by Andrew Hutchings (LinuxJedi), HP Advanced Technology Group
About LinuxJedi
• Worked for Sun/Oracle, MySQL Support Engineer• Worked at Rackspace, Drizzle Software Engineer• Worked at SkySQL, Senior Sustaining Engineer• Managed two teams, several projects at HP Cloud• Co-Author of MySQL 5.1 Plugins Development
About HP's Advanced Techonology Group
• An Open Source think-tank• Fosters collaboration throughout HP and Open Source communities
Other MySQL C Connectors
• libmysqlclient (GPLv2/proprietary)• libdrizzle (3-clause BSD)• Maria DB Client (LGPLv2.1 (sort-of))
None of these have an official fully non-blocking API!
Introducing libAttachSQL
libAttachSQL 1.0 Features
• Non-blocking API• Apache 2.0 license• Very lightweight (1.0.0 shared binary is 501KiB)• Can use OpenSSL and zlib for encryption and compression• Server side and (sort-of) local prepared statements• Good documentation from the start (http://docs.libattatchsql.org/)
Compatible With
• Ubuntu 12.04 and up• RedHat/CentOS 6.x and up• Windows (via. MinGW cross-compile)• GCC and CLang
Why not libdrizzle?
• libdrizzle 5.1 has a blocking API• Internal state loop is ugly to deal with• Networking code was hard to work with
Good things about libdrizzle
• Lightweight• Nice, simple API• Well tested (Jenkins CI)• Easy to contribute to• Liberal license
Why libAttachSQL?
• Modern techniques• All of the good points of libdrizzle, none of the legacy• Designed from the ground-up for use with MySQL
Event loops are web scale!
Non-blocking Internals
• Using libuv - a library born out of Node.js• Uses epoll/poll/kqueue in *nix• Uses I/O Completion Ports in Windows
Very Open Development
• GitHub (with issues & pull requests)• GitHub Pages + Pelican• Travis CI• Waffle.io• Read The Docs• Coverity Scan• IRC
Travis Tests
• GCC 32bit & 64bit• CLang 3.0 32bit & 64bit• CLang 3.4 64bit• No OpenSSL & zlib• Valgrind• Cppcheck• CLang Scan-Build• Documentation (Sphinx nit-pick mode)
Software Using libAttachSQL
• Sysbench• AttachBench
Basic Query Example
con= attachsql_connect_create("localhost", 3306, "test", "test", "testdb", NULL);attachsql_query(con, strlen(query), query, 0, NULL, &error);
Things to Note
• We haven't actually even connected yet!• No query/result object
while ((ret != ATTACHSQL_RETURN_EOF) && (error == NULL)){ ret= attachsql_connect_poll(con, &error); if (ret != ATTACHSQL_RETURN_ROW_READY) { continue; }
row= attachsql_query_row_get(con, &error); columns= attachsql_query_column_count(con); ... attachsql_query_row_next(con);}
Escaped Queries
const char *query= "SELECT * FROM t1 WHERE name = ? AND age > ?";attachsql_query_parameter_st param[2];
const char *name= "fred";uint32_t age= 30;param[0].type= ATTACHSQL_ESCAPE_TYPE_CHAR;param[0].data= (char*)name;param[0].length= strlen(name);param[1].type= ATTACHSQL_ESCAPE_TYPE_INT;param[1].data= &age;param[1].is_unsigned= true;attachsql_query(con, strlen(query), query, 2, param, &error);
Prepared Statements
const char *query= "SELECT * FROM t1 WHERE name = ? AND age > ?";
attachsql_statement_prepare(con, strlen(query), query, &error);while((ret != ATTACHSQL_RETURN_EOF) && (error == NULL)){ ret= attachsql_connect_poll(con, &error);}
const char *name= "fred";uint32_t age= 30;attachsql_statement_set_string(con, 0, strlen(name), name, NULL);attachsql_statement_set_int(con, 1, age, NULL);attachsql_statement_execute(con, &error);ret= ATTACHSQL_RETURN_NONE;while ((ret != ATTACHSQL_RETURN_EOF) && (error == NULL)){
ret= attachsql_connect_poll(con, &error); if (ret != ATTACHSQL_RETURN_ROW_READY) { continue; } attachsql_statement_row_get(con, &error); printf("ID: %d, ", attachsql_statement_get_int(con, 0, &error)); size_t len; char *name_data= attachsql_statement_get_char(con, 1, &len, &error); printf("Name: %.*s, ", (int)len, name_data); printf("Age: %d\n", attachsql_statement_get_int(con, 2, &error)); attachsql_statement_row_next(con);}
Connection Groups
• Group many connections into a single event loop• Many queries can run simultaneously on a single thread• Uses a callback API when there events to process• Can't be used for Prepared Statements (yet)
Connection Groups Example
group= attachsql_group_create(NULL);con[0]= attachsql_connect_create("localhost", 3306, "test", "test", "testdb", NULL);attachsql_group_add_connection(group, con[0], &error);attachsql_connect_set_callback(con[0], callbk, &con_no[0]);
attachsql_query(con[0], strlen(query1), query1, 0, NULL, &error);
while(done_count < 3){ attachsql_group_run(group);}attachsql_group_destroy(group);
void callbk(attachsql_connect_t *current_con, attachsql_events_t events, void *context, attachsql_error_t *error){ ... switch(events) {
case ATTACHSQL_EVENT_CONNECTED: printf("Connected event on con %d\n", *con_no); break;
case ATTACHSQL_EVENT_ERROR: printf("Error exists on con %d: %d\n", *con_no, attachsql_error_code(error)); attachsql_error_free(error); break;
case ATTACHSQL_EVENT_EOF: printf("Connection %d finished\n", *con_no); done_count++; attachsql_query_close(current_con); break;
case ATTACHSQL_EVENT_ROW_READY: row= attachsql_query_row_get(current_con, &error); ... attachsql_query_row_next(current_con); break;
Error Handling
attachsql_error_t *error= NULL;ret= attachsql_connect_poll(con, &error);if (error != NULL){ printf("Error occurred: %s", attachsql_error_message(error)); attachsql_error_free(error);}
Project Status
• 4 months in development (coding started 4th July)• 3 alpha releases, 2 beta releases, 1 RC release• First GA (1.0.0) is out now!• Packages for Ubuntu and RHEL/CentOS
Future
• Python wrapper• Other language wrappers• Server API?• Tools built around libAttachSQL
Questions?
Thank you
libAttachSQL - http://libattachsql.org/Freenode IRC - #libAttachSQLGitHub - http://github.com/libattachsql/libattachsql
Email [email protected] @LinuxJedi or @libAttachSQL