Frozen Perl 2010

The Perl Compiler

rurban - Reini Urban
Graz, Austria

What's new?

Who am I

rurban maintains cygwin perl since 5.8.8 and 3-4 modules, guts, B::* => 5.10

Mostly doing LISP Perl and PHP, and support for custom HW, windows + linux + real-time systems in real-life. Coding in winter, surfing in summer.

1995 first on CPAN with the perl5.hlp file and converter for Windows.

Contents

Started 1995 by Malcom Beattie, abandoned 2007 by p5p, revived 2008 by me

Very dynamic language. eval "require $foo;" -> which packages?

Why use B::C / perlcc?

Overview

In the Perl Compiler suite B::C are three seperate compilers:

perl toke.c/op.c - B::C - perl op walker run.c

Eliminate the whole parsing and dynamic allocation time.

The Walker

After compilation walk the "op tree" - run.c

  int
  Perl_runops_standard(pTHX)
  {
   dVAR;
   while ((PL_op = CALL_FPTR(PL_op->op_ppaddr)(aTHX))) {
      PERL_ASYNC_CHECK();
   }
   TAINT_NOT;
   return 0;
  }

The Walker

   while ((PL_op = CALL_FPTR(PL_op->op_ppaddr)(aTHX))) {
      PERL_ASYNC_CHECK();
   }

Observation

1. The op tree is not a "tree", it is reduced to a simple linked list of ops. Every "op" (a pp_<opname> function) returns the next op.

2. PERL_ASYNC_CHECK is called after every single op.

Perl Phases - the "Perl Compiler"

Normal Perl functions start at INIT, after BEGIN and CHECK.
The O modules start at CHECK, and skip INIT.

Perl Phases - the "B Compilers"

Perl Phases - the "B Compilers"

The B Compilers, invoked via O, freeze the state in CHECK, and invoke then the walker.

$ perl -MO=C,-omyprog.c myprog.pl
$ cc_harness -o myprog myprog.c
$ ./myprog

B::CC - Unoptimised / the walker

  while ((PL_op = CALL_FPTR(PL_op->op_ppaddr)(aTHX))) {
        PERL_ASYNC_CHECK();
  }

B::CC - The optimiser / unrolled

  while ((PL_op = CALL_FPTR(PL_op->op_ppaddr)(aTHX))) {
        PERL_ASYNC_CHECK();
  }
  is unrolled to:
  lab_15de5b8:
   PUSHMARK(sp);
   XPUSHs(GvSVn((GV*)PL_curpad[1]));
   PL_op = (OP*)&listop_list[0];
   PUTBACK; PL_op = pp_print(); SPAGAIN;
  lab_1248c18:
   PP_UNSTACK;
   PERL_ASYNC_CHECK();
  /* next basic block */

B::CC - The optimiser / unrolled




Status

5.6.2 and 5.8.9 non-threaded B::C are quite usable and have the least known bugs, but 5.10 and 5.12 became also pretty stable now.

Targets:

Status

Projects

Which software is compiler critical?

Projects

Which software is compiler critical?

Execution time is the same (sans B::CC)

Projects

Which software is compiler critical?

Execution time is the same (sans B::CC)

Startup time is radical faster

Projects

Which software is compiler critical?

Execution time is the same (sans B::CC)

Startup time is radical faster.

Web Apps with fast response times -

1 sec more or less => good or bad software

Projects

Which software is compiler critical?

Execution time is the same (sans B::CC)

Startup time is radical faster.

Web Apps with fast response times -

Optimise static initialization - strings and arrays

New Optimisations

Optimise static initialization - strings and arrays

non-threaded ! +10-20% performance

ltrace reveils Gthr_key_ptr, gv_fetchpv, savepvn, av_extend and safesysmalloc as major culprits, the later three at startup-time.

New Optimisations

Optimise static initialization - strings and arrays

non-threaded ! +10-20% performance

common constant strings with gcc -Os => automatically optimised

New Optimisations

Optimise static initialization - strings and arrays

non-threaded ! +10-20% performance

common constant strings with gcc -Os => automatically optimised

av_extend - run-time malloc => static arrays ?

New Optimisations

av_extend - run-time malloc => static arrays ?

static arrays are impossible if not Readonly

can not be extended at run-time, need to be realloc'ed into the heap.

New Optimisations

av_extend - run-time malloc => static arrays ?

pre-allocate faster with -fav-init or -O3

at least this is the idea. Same for hashes (nyi).

Real Life Applications

cPanel has used B::C compiled 5.6 for a decade, and wants to switch to 5.8.9 (or later).

cPanel offers web hosting automation software that manages provider data, domains, emails, webspace. A typical large webapp. Perl startup time can be too slow for many AJAX calls which need fast initial response times.

Benchmarks (by cPanel)

Larger code base => more significant startup improvements

Benchmarks (by cPanel)

Web Service Daemon
Resident Size (perlcc) 9072
Resident Size (perl) 9756

DNS Settings Client
Startup Time (perl) 0.074
Startup Time (perlcc) 0.021

HTML Template Processor
Startup Time (perlcc) 0.037
Startup Time (perl) 0.695

Plans

2010: Find and fix all remaining bugs

2010: Faster testsuite (Now 8 min - 40min - 2 days)

2011: CC type and sub optimisations

2012: CC unrolling => jit within perl (perl -j)

Emit parrot pir.

B::CC Limitations

run-time ops vs compile-time ...

dynamic range 1..$foo

goto/next/last $label

Undetected modules behind eval "require":
use -uModule to enforce scanning these

B::CC Limitations

run-time ops vs compile-time
BEGIN blocks only compile-time side-effects.

BEGIN {
   use Package; # okay
   chdir "dir"; # not okay.
   # only done at compile-time, not at the user
   print "stuff"; # okay, only at compile-time
   eval "what"; # hmm; depends
}

Move eval "require Package;" to BEGIN

B::CC Bugs

Custom sort BLOCK is buggy, wrong queue implementation

B::CC Bugs

Custom sort BLOCK is buggy, wrong queue implementation, causing an endless loop

sort { $a <=> $b }
is optimised away, ok

sort { $hash{$a} <=> $hash{$b} }
maybe?

sort { $hash{$a}->{field} <=> $hash{$b}->{field} }
for sure not

Testsuite

user make test (via cpan):

35x bytecode + c -O0 - O4 + cc -O0 - O2

=> 8 min

Testsuite

author make test:

35x bytecode + c -O0 - O4 + cc -O0 - O2 (8 min)

modules.t top100 (16 min)

+ testcore.t (16 min)

=> ~40 min

Testsuite

author make test 40 min

for 5-10 perls (5.6, 5.8, 5.10, 5.11 / threaded + non-threaded) 4*2=8

on 5 platforms (cygwin, debian, centos, solaris, freebsd)

=> 26 h (8*5*40 = 1600min) = 1-2 days, similar to the gcc testsuite.

Testsuite

top100 modules?

See webpage or svn repo for results for all tested perls / modules

With 5.8 non-threaded 3 fails Attribute::Handlers B::Hooks::EndOfScope YAML MooseX::Types

With blead non-threaded 4 fails Attribute::Handlers File::Temp ExtUtils::Install

unpredictable results: e.g. threaded 5.10 39/98 (cygwin release) vs 3/80 (a test version) fails. Innocent change => fatal consequences.

CC

CC - Type declarations

Currently:

my $EnameE_i; IV integer
my $EnameE_ir; IV integer in a pseudo register
my $EnameE_d; NV double


Future ideas are type qualifiers such as
my (int $foo, double $foo_d);

or attributes such as
my ($foo:Cint, $foo:Cintr, $foo:Cdouble);

or MooseX::Types

Code

http://search.cpan.org/dist/B-C/

http://code.google.com/p/perl-compiler/

Planned:

http://compiler.perl.org/

mailto:compiler@perl.org

Questions?