developing apps using perl
DESCRIPTION
Talk from PerlMova 2012 and YAPC::Russia "May Perl"TRANSCRIPT
Developing appsusing Perl
Anatoly SharifulinYAPC::Russia 2012
Hello, World!
«Happy Perl developer»about me by Andy Shitov
I program in Perl lessBut it doesn't mean I love it less :-)
JFDI became JFTIJust F*cking Talk about It
Success-storyof one app
And how Perl helped me/us with it
By us I mean Applifto
Our applications
Audience polling
«Why?» and «For what?» in details
http://www.slideshare.net/sharifulin/ss-12313103
A powerful reasonfor me
I was moving from LiveJournal to Posterous, used transfer scripts and changed Twitter and Facebook
crossposting settings when started to get replies like 'Tolya! Stop it! Enough!'
Current analogues are nice... not!
Slow, awkward, plain, trouble working:TwitWipe, TwitCide, Delete My Tweets, ...
«Stop managering, let’s start programming»
And have a talk at #yr2012 :-)
Approach and solutions
Main aims for me
• App must delete tweets quickly
• Appropriate deleting
• Cross-platform app: iOS, Android, Web, ...
Server API— hello, Perl!
All logic — on our server API (not Twitter API),clients is simple interface
Getting to know Twitter API
Little experience:
• Getting user profiles after login through Twitter on websites
• Auto-posting to Twitter with Net::Twitter(::Lite)
• AnyEvent::Twitter::Stream for real-time search (hey, @miyagawa!)
Blocking wayTask can be solved with Net::Twitter(::Lite)
LWP::UserAgent::POE Transport may be changed in Net::Twitter(::Lite)
NO
Idea — 2 separate asynchronous queues
Get timeline and delete tweets using REST Twitter API and asynchronous requests
This has to work fast!
Authorization and OAuth
Net::OAuth::AllWe've written our OAuth module long time ago,
supports all OAuth versions (1.0, 1.0A, 2.0)
Net::OAuth::Allhttps://github.com/likhatskiy/Net-OAuth-All
my $oauth = Net::OAuth::All->new( consumer_secret => $conf->{consumer_secret}, consumer_key => $conf->{consumer_key },
token => $data->{access_token }, token_secret => $data->{access_token_secret},);
$oauth ->via('GET') ->protected_resource_url('https://api.twiiter.com/...’) ->put_extra( include_rts => 'true', user_id => $user->{id}, count => 200, ) ->request('protected_resource');$oauth->to_header;$oauth->url; # $oauth->url_with_extra;
Patched the moduleUnauthorized user:
limit of 150 requests per hour from one IP, not 350 requests per hour from one user
Asynchronous requests
I use MojoliciousNot a secret at all :-)
Mojo::UserAgentMojo::IOLoopGood asynchronous HTTP client
my $delay = Mojo::IOLoop->delay;for (@$tasks) { ... $delay->begin; $ua->get($url => {'Authorization' => $h} => sub { my ($ua, $tx) = @_; ... $delay->end( ... ); });}
say $delay->wait;
Server API
Server APIAPI supports GET and POST requests,
JSON transmission format,response codes 200 and 50x.
Server APIStarman + Mojolicious
Mojolicious::Plugin::ApiHelpers
package App::Api;use App::Base -controller, with => [ 'App::Task', 'App::User' ];
sub error { shift->api_error(code => 1) }sub any { shift->api_error(code => 5) }
...
1;
my $api = $r->route('/api')->to('api#', api => 1);$api->route('/:action', action => qr/login|oauth/)->to;
my $apiu = $api->bridge->to('#auth', sign => 1);$apiu->bridge('/task/new')->to('#task_check') ->route->to('#task_new');
$api->bridge('/')->to('#auth', maybe => 1, sign => 0) ->route->to('#hello');
10 methods,123 tests and documentation
Most methods require user authorization and check sign (api_id + secret)
Test::MojoTest::More
Testing API methods access, input-output data, signature check and etc.
Test::MojoTest::More
A real task from a test user
All methods are coveredWithout Mock-objects, no check
if a correct cluster of tweets have been deleted
Too laborious and needlessA real cycle cannot be tested:
posting tweets – deleting – check
$t->get_ok("$url/") ->status_is(200) ->header_is('Access-Control-Allow-Origin', '*') ->json_content_is({hello => 'Hello DLTTR!'});
$t->get_ok(sign get => "$url/task/new") ->status_is(200) ->json_content_is({error => {msg => 'User authorization failed', code => 2}});
Too risky to test on your own account :-)
— You don't care about you Twitter acc, do you?— Yep!— So I can delete all your tweets?— Nooooooooooooooo!!!
Simple debug reguests
To check the quality of client work requests/responses with before_dispatch + after_dispatch may be used
Three queues
1. Queue for timelinesSearch and filters on tweets ID to delete,
paging with since_id and max_id
1. Queue for timelinesMax. 200 tweets per request,
getting timeline one by one per user, Rate Limit check
«Limits» by TwitterNo access to all tweets,
only the latest ~3200 or starting from any exact date, counter is not on zero and etc.
2. Queue for deleting tweets
Deleting with 'clusters' of 200 tweets, no limits. «Buster» or «Cannon» :-)
2. Queue for deleting tweets
Error processing — Twitter API error doesn't always mean a failure
3. Queue for Push notify
On iOSNet::APNS::Persistent (also AnyEvent::APNS),
to check if tokens are valid — Net::APNS::Feedback
On AndroidWWW::Google::C2DM and WWW::Google::ClientLogin
3. Queue for Push notify
Database
So easy — mysqlStoring and task orders,
just to relax :-)
2 finer points:
1. Create correct indexFor queues
2. Disable request caching
select SQL_NO_CACHE * from task where ...
Site and web-version
Site and web-version
• Starman + Mojolicious
• Mojolicious::Plugin::I18N2— my fork of I18N plugin
• Mojolicious::Plugin::OAuth— login through Net::OAuth::All
• Mojolicious::Plugin::ShareHelpers
• Mojolicious::Plugin::UtilHelpers
Admin and statistics
«Big Brother»All details on users, number of tasks, feedbacks
and real-time statistics
Statistics on users
Users wall
«Ordinary» apps don't have statistics like that
1M+ tweets deleted,3K+ users,
20% make purchaseFor 2 months
Promotion
Promotion on TwitterFollowing, favorites, replies and search —
Net::Twitter(::Lite)
But they blocked me three times already :-)
Promotion on Twitter
Summary
Positive
• Easy solution using pure Perl(work time is about 30 hours)
• Works fast, really fast
• Right delete tweets
• Any changes, updates, new services— everything's on the server, no hard
• Real-time statistics
Only one negative:Server's down — the app doesn't work,
users become mad, low rates and spell curses :-)
Check out DLTTR!http://dlttr.com/app http://dlttr.com/android
www.dlttr.com
«Seems like a simple app but in fact it's awesome
and in Perl»
P. S.
At #BarCampKrrI had a talk
«Developing app from idea to release»
(Perl wasn't mentioned)
But everything came down to... Perl
«What language do you use?», «But why?»,
«Is it difficult to find programmers like those?» :-)
use Perl or die;
Thanks!Anatoly SharifulinYAPC::Russia 2012