the perl api for the mortally terrified (beta)

The Perl APIfor the

Mortally Terrified

Mike Friedman(friedo)

London Perl WorkshopNovember 30, 2013

MeMike Friedman

Mike Friedman(friedo)

This talk is the product of a lengthy journey.

It's also a work in progress.

I have some limited experience in integrating Perl and C with XS, and would enjoy the opportunity to expand my skills in this area.

i R dum.

- Learned from this book.

- Learned from this book.- It’s very good

- Learned from this book.- It’s very good- The printing I got was messed up.

- Learned from this book.- It’s very good- The printing I got was messed up.- Like every " was replaced by ©

The Perl APIWhat’s it good for?

Make things fast:

Make things fast: - Write your slow Perl code in C

Make things fast: - Write your slow Perl code in C - Keep your fast Perl code in Perl

Make Perl more useful:

Make Perl more useful: - Interface with libraries written in C

Make Perl more useful: - Interface with libraries written in C - Create modules exposing external APIs to Perl

Make Perl do...stuff

Make Perl do...stuff - Change opcodes on the fly?

Make Perl do...stuff - Change opcodes on the fly? - Extend the parser?

Make Perl do...stuff - Change opcodes on the fly? - Extend the parser? - Make exceptions work?

Make Perl do...stuff - Change opcodes on the fly? - Extend the parser? - Make exceptions work? - Why not?

A Crash Course in

C Crash Course

Basic Types (there are more)

C Crash Course

char foo = ‘a’;

C Crash Course

char foo = ‘a’;int bar = 42;

C Crash Course

char foo = ‘a’;int bar = 42;long baz = 198547;

C Crash Course

char foo = ‘a’;int bar = 42;long baz = 198547;float quux = 3.14159;

C Crash Course

char foo = ‘a’;int bar = 42;long baz = 198547;float quux = 3.14159;double splat = 3.14159265358979;

C Crash Course

Pointers

C Crash Course

Pointers

char *foo; /* ptr to char */

C Crash Course

Pointers

char *foo; /* ptr to char */int *bar; /* ptr to int */

C Crash Course

Pointers

char *foo; /* ptr to char */int *bar; /* ptr to int */long *baz; /* ptr to long */

C Crash Course

Pointers

char *foo; /* ptr to char */int *bar; /* ptr to int */long *baz; /* ptr to long */float *quux; /* ptr to float */

C Crash Course

Pointers

char *foo; /* ptr to char */int *bar; /* ptr to int */long *baz; /* ptr to long */float *quux; /* ptr to float */double *splat; /* ptr to double */

C Crash Course

Pointers

char *foo; /* ptr to char */int *bar; /* ptr to int */long *baz; /* ptr to long */float *quux; /* ptr to float */double *splat; /* ptr to double */void *fnarf; /* ptr to something */

C Crash Course

Pointers are integer variables which hold a memory address.

C Crash Course

C Perl

C Crash Course

int i = 42; my $i = 42;

C Perl

C Crash Course

int i = 42; my $i = 42;int *i_ptr = &i; my $i_ref = \$i;

C Perl

C Crash Course

Read unary & as “address-of.”

C Perl

C Crash Course

int j; my $j;

C Perl

C Crash Course

int j; my $j;j = *i_ptr; $j = $$i_ref;

C Perl

C Crash Course

int j; my $j;j = *i_ptr; $j = $$i_ref;

Read unary * as “dereference.”

C Perl

C Crash Course

Aggregates

C Crash Course

Aggregates

int foo[10]; /* array of ints */

C Crash Course

Aggregates

int foo[10]; /* array of ints */float bar[12]; /* array of floats */

C Crash Course

Aggregates

int foo[10]; /* array of ints */float bar[12]; /* array of floats */char baz[42]; /* a string! */

C Crash Course

Aggregates

C Crash Course

Aggregatesstruct person { int age; char *name; double acct_balance;};

C Crash Course

Aggregatesstruct person { int age; char *name; double acct_balance;};

age *name acct_balance

C Crash Course

Aggregates

C Crash Course

Aggregatesunion thingy { long num; char chars[4]; foo_t *myfoo;};

C Crash Course

Aggregatesunion thingy { long num; char chars[4]; foo_t *myfoo;};

f3 de 42 9a

thingy.num

thingy.chars

thingy.myfoo

C Crash Course

Using structs

C Crash Course

struct person the_dude;the_dude.age = 42;the_dude.name = “jeffrey lebowski”;the_dude.acct_balance = 1.79;

Using structs

C Crash Course

struct person the_dude;the_dude.age = 42;the_dude.name = “jeffrey lebowski”;the_dude.acct_balance = 1.79;

Using structs

00 2a 3f 39 d2 90 a4 70 3d 0a d7 a3 fc 3f

age *name acct_balance

j e f f r e y ...

C Crash Course

Typedefstypedef struct { int age; char *name; double acct_balance;} person;

person the_dude;

What is XS?

perldoc perlxs:

XS is an interface description file format used to create an extension interface between Perl and C code (or a C library) which one wishes to use with Perl. The XS interface is combined with the library to create a new library which can then be either dynamically loaded or statically linked into perl. The XS interface description is written in the XS language and is the core component of the Perl extension interface.

XS is a specialized templating language for C.

Some XS Code

This stuff is XS.

Some C Code made by xsubpp

XS != Perl API

For most things

You don't actuallyhave to use XS.

(OK. You don’t have to know that you’re using it.)

Inline::C

use Inline C;hello('Mike');hello('Cookie Monster');

__END__

__C__void hello(char* name) { printf("Hello %s!\n", name);}

use Inline C;hello('Mike');hello('Grover');

__END__

Inline::C Code in Perl

Digest::MD5

C Code Parse::RecDescent

Digest::MD5

Create build in a temp dir

Digest::MD5

Write XS

Digest::MD5

Write XS

Compile and install

Digest::MD5

Write XS

Compile and install

How do we make Perl do stuff in C?

We need to understand Perl’s data structures.

Handwaving Ahead

A $scalar in Perl is represented by a struct called SV.

SvNULL

SvRV SvNVSvPV SvIV

(These are not all of them.)

What does SV look like?

sv_any sv_refcnt sv_flags

flags type

What does SV look like?

flags type

If this scalar is undef, then sv_any is NULL

What does SvPV (a string) look like?

xpvpvx current len alloc len

a s t r i n gchar* \0

What do SvIV and SvNV look like?

pvx current len alloc len

3 1 3 3 7 \0char*

69 7a 00 00

What do SvIV and SvNV look like?

pvx current len alloc len

3 . 1 4 1 5char*

6e 86 1b f0

ivxf9 21 09 40

WRITE CODE NOW?!??!!11!?

Let's make something fast.

#!/usr/bin/env perluse strict;use warnings;use Inline 'C';

print 'Give me a number: ';my $num_a = <STDIN>;

print 'Give me another number: ';my $num_b = <STDIN>;

printf "The sum is: %s\n", add( $num_a, $num_b );__END____C__

int add( int a, int b ) { return a + b;}

bash-3.2$ perl cexp1.plGive me a number: 42Give me another number: 11The sum is: 53

Run it!

bash-3.2$ perl cexp1.plGive me a number: 42Give me another number: 31.337The sum is: 73

Run it again!

bash-3.2$ perl cexp1.plGive me a number: 42Give me another number: 31.337The sum is: 73

Run it again!

What happened?

Inline::C is

CRAZY SMART

intintintint

Inline::C is

CRAZY SMART

use Inline 'C';

print 'Give me a number: ';my $num_a = <STDIN>;print 'Give me another number: ';my $num_b = <STDIN>;$num_a += 0; $num_b += 0;printf "The sum is: %s\n", add( $num_a, $num_b );__END____C__SV *add( SV *a, SV *b ) {

if ( SvIOK( a ) && SvIOK( b ) ) { return newSViv( SvIV( a ) + SvIV( b ) ); } else if ( SvNOK( a ) && SvNOK( b ) ) { return newSVnv( SvNV( a ) + SvNV( b ) ); } else { croak( "I don't know what to do!" ); }}

use Inline 'C';

bash-3.2$ perl cexp2.pl Give me a number: 42Give me another number: 11The sum is: 53bash-3.2$ perl cexp2.pl

Run it!

bash-3.2$ perl cexp2.pl Give me a number: 53.4Give me another number: 6.54The sum is: 59.94

Run it!

bash-3.2$ perl cexp2.pl Give me a number: 53.4Give me another number: 6.54The sum is: 59.94

Give me a number: 42Give me another number: 6.54I don't know what to do! at cexp2.pl line 16, <STDIN> line 2.

Once you get this far, everything you need

is in perlapi.

Let's do something with arrays.

What does an AV look like?

array fill max

alloc array len*some fields omitted

SV ...

print "Give me some numbers: ";my @numbers = map { $_ += 0 } split /,/, <STDIN>;

my $result = squares( \@numbers );printf "The squares are: %s\n", join ", ", @$result;

__END__

print "Give me some numbers: ";my @numbers = map { $_ += 0 } split /,/, <STDIN>;

my $result = squares( \@numbers );printf "The squares are: %s\n", join ", ", @$result;

__END__

SV *squares( SV *numbers ) { AV *array = (AV *)SvRV( numbers ); AV *return_array = newAV(); SV **tmp;

int len = AvFILL( array ); int i, val;

for( i = 0; i <= len; i++ ) { tmp = av_fetch( array, i, 1 ); if( !SvIOK( *tmp ) ) { croak( "Can't handle this value!" ); } val = SvIV( *tmp ); val = val * val;

av_push( return_array, newSViv( val ) ); }

return newRV_inc( (SV *)return_array );}

return newRV_noinc( (SV *)return_array );}

bash-3.2$ perl cexp3.pl Give me some numbers: 4,5,6,23The squares are: 16, 25, 36, 529

Run it!

Let's do something with hashes.

What does an HV look like?

(Diagram missing

(So I stole this one))

(Diagram missing

(So I stole this one))

print "Give me some words: ";my @words = split /,/, <STDIN>;chomp @words;my $result = uniques( \@words );printf "The uniques are: %s\n", join ", ", %$result;

__END__

SV *uniques( SV *words ) { AV *array = (AV *)SvRV( words ); HV *result = newHV(); SV **tmp;

int len = AvFILL( array ); int i; char *val;

for( i = 0; i <= len; i++ ) { tmp = av_fetch( array, i, 1 ); if( !SvPOK( *tmp ) ) { croak( "Can't handle this value!" ); }

val = SvPV_nolen( *tmp );

hv_store( result, val, strlen( val ), newSV(0), 0 ); }

return newRV_noinc( (SV *)result );}

bash-3.2$ perl cexp4.pl Give me some words: foo,bar,baz,baz,foo,quux,narf,poitThe uniques are: bar, narf, baz, poit, quux, foo

Run it!

What isMagic?

Not that terrifying.

Not that terrifying.(code sample missing)

Magic is a linked list of function pointers attached to a *SV.

The functions implement custom read/write behavior.

Magic is a linked list of function pointers attached to a *SV.

The functions implement custom read/write behavior.

Special vars like %ENV are implemented via Magic.

$ENV{foo} = "blah";

assignment op

$ENV{foo} = "blah";

Magic on %ENV ?

assignment op

$ENV{foo} = "blah";

Magic on %ENV ?

assignment op

Look up write function

$ENV{foo} = "blah";

Magic on %ENV ?

assignment op

Look up write function

call libc setenv()

tie() is also implemented via Magic.

Terror defeated!

Where do we go from here?

Resources:

perlgutsFor details on how to manipulate Perl's

data structures.

Resources:

perlapiFor a comprehensive reference to (almost)

everything you need to make perl do stuff.

Resources:

Perlguts Illustrated

For pretty diagrams and explication.

Resources:

Inline::C Cookbook

For common solutions to problemsinterfacing between Perl and C

Resources:

Code samples

http://github.com/friedo/perl-api-terror

London Perl Workshop

Thank you.

Mike Friedmanfriedo@friedo.com

http://friedo.com/http://github.com/friedo/perl-api-terror

the perl api for the mortally terrified (beta)

c crash coursecperlint

c crash courseusing

c crash coursebasic

c crash coursec int

ptr ptr ptr ptr ptr

c crash coursetypedefs

perl api whats

int age char

Technology

1 introduction to perl and perl syntax learning perl,...

perl 4: applied perl - o’reilly...

transmission lines for the terrified jeff 9v1as...

bricks + mobile 2011 - terrified retailers take control

computers for the terrified

tcgp conference - elearning for the terrified

4 surprised / terrified / confused grammar 5 unhappy

telesales for the terrified

statistics for the terrified

perl switches with the perl command

maxthon and perl(beijing perl workshop 2009)

the perl api for the mortally terrified

perl power tools - saint perl 6

perl/dbi - accessing databases from perl

programming perl - perl training · this is version 1.39 of...

perl 101 - the basics of perl programming

perl perl 24 hours

readme - ibm · 2016. 7. 1. · openldap openssh...

i. mengenal perl - gapra.files. · pdf filei. mengenal perl...

perl::critic and perl::tidy