Fabien Potencier

Symfony Live Conference, Symfony 2.0, and Dependency Injection

2010-02-19T10:02:00+01:00

This has been an amazing week for me, Symfony, and the whole Symfony community. With more than 350 attendees coming from more than 30 countries, the Symfony Live conference was a blast. I want to thank all attendees, speakers, and the Sensio Labs team for coming to the conference and making it a truly fantastic event.

If you have not attended the conference, you can still come to the Symfony Live After-Party next week in London. And don't miss Symfony Camp in June and Symfony Day in October. If you live far away from Europe, stay tuned as we have more exciting news to announce in the coming weeks.

The conference was also the occasion to unveil the first preview release of the upcoming Symfony 2 framework. Since then, the buzz has been great and I have already received a lot of good comments on it. Of course, I hope that this is just the beginning, and that more people will provide feedback.

On the PHP Community

I'm also very proud that people from so many different PHP "sub-communities" attended the conference: Jon Wage (Doctrine project manager and lead developer of Doctrine 1.x), François Zaninotto (lead developer of Propel), Nils Adermann (lead developer of PhpBB), and of course, Matthew Weier O'Phinney (Zend Framework lead).

I like to see the whole PHP community as a big unified group of developers trying to promote the same platform. And I do my best to be a good "PHP citizen". I try to innovate by adapting concepts from other languages. But I also try to not reinvent the wheel when it's not mandatory. The symfony 1 framework uses a lot of third-party libraries like Propel and Doctrine. And for Symfony 2, I'm proud to have replaced our own Logger and Cache system by the equivalent components from the Zend Framework. And you can imagine that it was not an easy decision to take. First, because it meant throwing away code we have written and maintained for years. Then, because the Zend Framework is our main competitor. But I'm sure we have made the right decision and Matthew seems to have the same point of view on that matter.

On PHP Frameworks

People are jealous, so jealous that it makes me really sad. For the last few years, some PHP framework communities attempted to "kill" Symfony by trying to demonstrate that it is too slow, too complex, and too bloated. Of course, that's not true. And the number of high-traffic websites using symfony 1 speaks for us.

I get inspiration from many different frameworks from many different languages. Besides PHP projects, I like to follow the development of Django, Spring, Rails, Maven, Jinja, Rack, Sinatra, and many others. I try to bring innovation to the table. And for Symfony 2, I have tried to take into account the major pain points of symfony 1. The truth is that Symfony 2.0 requires no configuration. The truth is that Symfony 2 is really fast. Is it the fastest framework? I don't care. It is probably fast enough for your next website, and so is symfony 1.

Now if some people cannot accept that Symfony 2 is really fast and if they cannot accept that Symfony 2 is great step forward for PHP, that's sad. The web evolves very fast. Competition is everywhere for PHP. We should all be in the same boat.

I would love if people from different framework communities can work together more often, like what we have done with the PHP 5.3 interoperability group. I would love if we can share more components. I would love to discuss how we can make our PHP community grow faster.

Is it a dream? I hope it's not. And I have a proposal. Let's organize an event where several PHP framework communities can discuss and share ideas. Anyone?

On Dependency Injection

The biggest Symfony 2 innovation lies in its low level architecture. And its flexibility and speed is mainly due to the Symfony Dependency Injection Container component. Dependency Injection Containers are not widespread in the PHP world, and I really think that this is a very good approach. That's why I talk about Dependency Injection at conferences. Next time I will talk about this topic is during the ConFoo conference in March.

The state of YAML in PHP

2009-12-21T08:20:00+01:00

My first exposure to YAML was in 2001, back in the days when I was mainly working with Perl. Well, I was not using YAML per se at that time, but rather Data::Denter, a Perl library that provides data serialization/deserialization. I used this library mainly for debugging purposes. From its documentation:

"It formats nested data structures in an indented fashion. It is optimized for human readability/editability, safe deserialization, and (eventually) speed."

At the end of the year 2002, the module was deprecated in favor of a new serialization language, YAML, with the added bonus of being programming language independent. I promptly switched to use the Perl YAML module, and I never looked back. I used YAML as a mean to debug my Perl programs, but I also started to use it more and more to store configuration data.

When I started to use PHP at the end of 2004, one of the first thing that quickly bothered me was the poor support for YAML in the PHP world.

By the way, if symfony uses YAML a lot, it has nothing to do with Ruby on Rails ;) It just happens that Ruby also has some Perl heritage!

But first, what is YAML?

According to the official YAML website, YAML (YAML Ain't Markup Language), is a human friendly data serialization standard for all programming languages.

YAML can be used to describe both simple and complex data structures. It's an easy to learn language that describes data. As PHP, it has a syntax for simple types like strings, booleans, floats, integers, arrays, and even more complex ones like objects.

Nowadays, YAML is a heavily used format for configuration files, mainly because even non programmers are able to understand and modify YAML files easily.

To sum up the benefits of YAML, I often say that YAML files are as expressive as XML files and as readable as INI files.

Since the creation of YAML, another lightweight data-interchange format has come to life: JSON. JSON is quite similar to YAML (and as a matter of fact, JSON is a subset of YAML); but even if it is easy for humans to read and write, I think it is not as readable as YAML, and a bit too verbose.

YAML

If you already know what is YAML and how to use it to describe your data structures, just skip this section.

Besides strings, Booleans, and numbers, let's have a look at one of the simplest configuration structure you can describe with YAML:

key: value
foo: bar

The above snippet is the simplest way to express key/value pairs in YAML. The foo key has a bar value. The equivalent PHP code would be:

array('key' => 'value', 'foo' => 'bar')

And that's pretty much covers what you can do with ini files. Speaking of ini files, you can also group key/values under "sections". Here is how this is possible with YAML:

section1:
  foo: bar
 
section2:
  bar: foo

The equivalent PHP code reads as follows:

array(
  'section1' => array('foo' => 'bar'),
  'section2' => array('bar' => 'foo'),
)

That does the trick because there is several ways to describe key/value pairs. The short notation (foo: bar), and the expanded one, where you use indentation to describe nested structures as above.

The same data structure can also be described as follows:

section1: { foo: bar }
section2: { bar: foo }

The {} is how you enclose a hash. That's one of the greatest benefit of YAML as a description format: you can visually organize your data by using one of the three possible notations.

Unlike PHP, YAML makes a difference between hashes (mappings) and arrays (sequences):

[1, 'a string', "another string"]

The above snippet, a YAML sequence, is the equivalent of the following PHP code:

array(1, 'a string', "another string")

If you mix and match mappings and sequences, short and verbose notations, you can describe very complex data structures:

section1:
  foo: { bar: foo }
  bar: [1, 2]
  foobar:
    - 'a string'
    - 'another one'

This section has barely scratched the surface of what you can express with YAML. If you want to learn more, you will find plenty of documentation on the Internet.

YAML in PHP

YAML is human-friendly, but not so developer-friendly for someone willing to write a parser for it. The YAML specification is really huge. If you read it carefully, you can easily imagine that writing a YAML parser is not an easy task. As I mainly use YAML as a configuration format like many other developers, I'm more looking for a fast, incomplete but correct library, instead of a fat, spec-compliant one.

Back in 2005, I was looking for such a YAML parser and dumper for PHP. Chris Wanstrath, who will eventually create Github some years later, wrote one such limited parser and dumper, Spyc, specifically to be used as a simple configuration library.

I used it for symfony 1.0. I fixed some bugs from time to time, but as time passed, I found many limitations and became more and more frustrated about it. One day, I eventually decided to write a more robust and stable YAML parser and dumper for symfony.

Since then, Alexey Zakhlestin created a PECL extension that wraps the Syck library.

At the beginning of 2009, I decided to release this library as a standalone library, with no dependency whatsoever. It means that you can start using it today.

The YAML Symfony Component

Released under the MIT license, the YAML Symfony Component can be used in any application, even commercial ones.

When I created this YAML library for PHP, I had several goals in mind:

Ease of use: Installation should be easy and fast. Install it via PEAR, download an archive, or checkout the SVN or Git repository, and you are ready to go. No configuration. Drop the files in a directory and start using it right away.
Fast: One of the main goal of Symfony YAML was to find the right balance between speed and features.
Unit tested: The library is unit-tested (with more than 400 unit tests as of today).
"Real" Parser: To correctly handle a large subset of the YAML specification, a dedicated and hand-written parser has been written. The parser is robust, easy to understand, and simple enough to extend.
Clear error messages: Whenever you have a syntax problem with your YAML files, the library should output helpful messages with the filename and the line number where the problem occurred. It eases debugging a lot.

And of course, YAML being not so well-known in the PHP world, the YAML component also comes with a full documentation.

The easiest way to install the Symfony YAML Component is probably to use the PEAR installer:

$ pear channel-discover pear.symfony-project.com
$ pear install symfony/YAML

Using YAML in your Projects

The Symfony YAML library consists of two main classes: one to parse YAML strings, and the other to dump a PHP variable to a YAML string. On top of these two core classes, the main sfYaml class acts as a thin wrapper and simplifies common uses:

// loading a YAML file or a YAML string
$var = sfYaml::load('/path/to/file.yml');
 
// Dumping a PHP variable to YAML
$yaml = sfYaml::dump($var, $inline);

YAML for PHP 5.3

The previous sections use the PHP 5.2 compatible version of the library. If you have already switched to use PHP 5.3, the good news is that the YAML Component is already available for that version too. For now, it is only available on the Symfony 2 Subversion repository:

$ svn co http://svn.symfony-project.com/branches/2.0/lib/Symfony/Components/YAML/ YAML

use Symfony\Components\YAML\YAML;
 
// loading a YAML file or a YAML string
$var = YAML::load('/path/to/file.yml');
 
// Dumping a PHP variable to YAML
$yaml = YAML::dump($var, $inline);

This version can be autoloaded with any autoloader that follows the standards discussed by some PHP developers. Symfony 2 provides such an autoloader:

require_once __DIR__.'/lib/Symfony/Foundation/ClassLoader.php';
 
use Symfony\Foundation\ClassLoader;
 
$loader = new ClassLoader('Symfony', __DIR__.'/lib');
$loader->register();

The YAML Symfony Component is already used by and bundled with many popular Open-Source PHP software like symfony, Doctrine, and PHPUnit. Other frameworks like the upcoming Okapi2 framework and the mootools plugins repository, announced some days ago, make a heavy use of YAML and also use the YAML Symfony Component.

Next time you look for a flexible mean to store or share data, consider using YAML!

2009 Conferences Wrap-up

2009-12-15T11:09:00+01:00

This year, I traveled in 7 countries for PHP conferences, giving a total of 12 talks. It has been a very exiting year of conferences for me. I met old and new friends, I learned a bunch of new things (not always related to PHP) and I had of course very interesting conversations about PHP 5.3, symfony, Twig, the Zend Framework, and various other topics.

In March, I went to the PHP Quebec conference. Three years ago, this was my first PHP conference, and as such it's always a very special event for me. This was also the first conference where I gave a talk about Symfony 2.

In May, I had the pleasure to go to the PHP Day in Italy. This was probably one of the most convivial conference of the year. Italian people are very friendly, and the food is always amazing.

In June, it was the first Symfony Live conference. I was one of the organizer, and also a speaker of course. This was a great success and I had a wonderful time talking with so many symfony users.

In September, I went to Japan for their 10th PHP conference. This was my first time in Japan, and I'm eager to come back someday. This was also the first time I worked with a professional translator.

In October, thanks to Zend, I went to the Zend PHP Conference in San Jose. Needless to say that this was an amazing experience. I have also kind of officially "launched" Twig at this conference, and created Pirum during a night thanks to the jetlag. This is also where I decided that Symfony 2 would only support PHP 5.3. At the end of October, I also went to the PHP Barcelona conference for the first time.

In November, after the Forum PHP conference in Paris, where I talked about PHP 5.3, my new friend, I went to the IPC conference in Karlsruhe. Two great conferences, where I always meet a lot of friends.

Thanks to all conference organizers for giving me these great opportunities.

All my slides are available in the talk section of this website, and also on Slideshare.

Here is a list of the most interesting/up-to-date talks I gave this year:

2009 is coming to an end, and 2010 is already almost here. I'm going to be doing a fair bit of traveling early next year as I will speak at least at these conferences:

Dependency Injection in PHP 5.2 and 5.3: PHPBenelux conference (January 30, 2010 - Antwerp, Belgium)
Playing with the symfony components: Symfony Live Conference (February 16, 2010 - Paris, France)
Symfony 2 revealed: Symfony Live Conference (February 17, 2010 - Paris, France)
PHP 5.3 in practice: PHP UK Conference (26 February, 2010, London, England)
Symfony Components: What's in for you?: ConFoo (March 10, 2010, Montreal, Canada)
Dependency Injection in PHP 5.2 and 5.3: ConFoo (March 10, 2010, Montreal, Canada)

See you next year at one of the PHP conference!

Pirum, the Simple PEAR Channel Server Manager

2009-11-28T08:50:00+01:00

Some weeks ago during the Zend Conference, I quietly released Pirum, a simple PEAR channel server manager. As some people talk about it on "social networks", I thought I should write an official announcement on my blog to explain where I come from.

A PEAR channel server allows you to install PEAR packages with the PEAR command line. You are probably already familiar with it as it comes bundled with most PHP installations:

$ pear install ...

Pirum comes from my frustration with the current state of PEAR channel servers. Beside big Open-Source projects like symfony, I'm also responsible for smaller projects (like Twig, or Swift Mailer). But for all my software, I like to provide different way of installing them: directly from SVN via svn:externals, from my Git mirrors, from an archive to download, or with the PEAR command line.

Except for the PEAR package installation, the other ones are quite easy to setup and automate. But providing a PEAR channel proves to be more involving. The only serious PEAR channel server I'm aware of is Chiara_PEAR_Channel from Greg Beaver. This is the software I used for symfony for four years. It works really well, but is a bit cumbersome to setup for smaller projects (it needs a database, and you need to configure some categories, the contributors for the project, and more). That's fine for PEAR itself, but frankly, for the rest of us, that's just too much.

As a matter of fact, hosting a PEAR channel server is super simple. It's all about static files. That's right, even if you need some sort of PHP scripts to maintain a PEAR channel server, the frontend used by the PEAR command line tool can only be static files (mainly XML ones). And Chiara_PEAR_Server works exactly like this. The backend, done in PHP, is where you maintain your packages, and it generates the frontend, a bunch of static files for the frontend. That's a really great architecture as it allows to scale very easily. For instance, you should probably be able to host a PEAR channel server on a CDN line Amazon S3 in a matter of minutes.

Thanks to this decoupled and simple architecture, I wrote some PHP scripts to manage a PEAR channel server for the symfony plugins two years ago.

The idea for Pirum came when I started the Twig project. Obviously, I wanted a PEAR channel server for Twig, but I really did not want to use Chiara_PEAR_Server. So I started to hack my own and Pirum was born.

Pirum lets you setup PEAR channel servers in a matter of minutes. Pirum is best suited when you want to create small PEAR channels for a few packages written by a few developers.

Pirum consists of just one file, a command line tool, written in PHP. There is no external dependencies, no not need for a database, no need to setup credentials, and nothing need to be installed or configured.

Installing Pirum is as simple as downloading the pirum file and saving it where you see fit.

Of course, Pirum itself uses Pirum to provide a PEAR channel server for itself!

Even if Pirum only consists of just one file, it comes with a lot of great features:

Besides its size and simplicity, Pirum is packed with a lot of features:

It creates a full-featured PEAR channel server useable by any PEAR CLI;
Each channel has an HTML page describing the server and the packages it hosts;
New releases can be tracked by subscribing to an Atom feed.

Of course, it also comes with some limitations:

No support for fallback PEAR channel servers;
No category management (all packages are under a "default" category);
No web interface for managing the packages.

You can find more information about using Pirum on its official website.

Pirum is already used by the following Open-Source projects:

I hope that with Pirum, more Open-Source projects will start providing a PEAR channel as a mean to install their software. If you start using Pirum for your project, please send me an email so that I can update the list on the Pirum website.

PHP 5.3.0 on Mac OS 10.6 (Snow Leopard)

2009-11-05T09:40:00+01:00

Recently, I started to use PHP 5.3.0 extensively, thanks to the move of Symfony 2 to PHP 5.3. To do my work, I need to have several different versions of PHP at hand, with different configurations, and so I always compile my own PHP and Apache binaries on my Mac. But as Snow Leopard comes bundled with PHP 5.3.0, I started some PHP 5.3.0 experiments with the bundle version and everything went well for a couple of days.

Now that I use PHP 5.3.0 more and more, I decided it was time to compile my own version. As it turns out, compiling PHP 5.3.0 on Snow Leopard is a bit involving. As I loose some time before finding the right solution, here is what I did step by step (I have just followed the tips referenced in the ticket opened for the issue on php.net, so all credits go to them).

First, I ran the configure script with my usual options and added --with-iconv-dir=/usr at the end:

$ ./configure ... --with-iconv-dir=/usr

If you have upgraded your Mac to Snow Leopard, you should know that everything must now be compiled for 64 bits. It means that you might need to recompile extra libraries needed by PHP like libjpeg or libpng, or even re-install a 64 bits version of MySQL.

I then patched the iconv.c file the same way Apple did for their own version:

--- php-5.3.0/ext/iconv/iconv.c 2009-03-16 22:31:04.000000000 -0700
+++ php/ext/iconv/iconv.c   2009-07-15 14:40:09.000000000 -0700
@@ -51,9 +51,6 @@
 #include <gnu/libc-version.h>
 #endif
 
-#ifdef HAVE_LIBICONV
-#undef iconv
-#endif
 
 #include "ext/standard/php_smart_str.h"
 #include "ext/standard/base64.h"
@@ -182,9 +179,6 @@
 }
 /* }}} */
 
-#ifdef HAVE_LIBICONV
-#define iconv libiconv
-#endif
 
 /* {{{ typedef enum php_iconv_enc_scheme_t */
 typedef enum _php_iconv_enc_scheme_t {

To apply the patch, save the above text in a file (patch) and from the root source of PHP, type the following command:

$ patch -p1 < patch

Then, add -lresolv to the EXTRA_LIBS variable:

$ export EXTRA_LIBS=-lresolv

Eventually, run the usual make commands

$ make
$ make install

My first Zend Conference was a blast

2009-10-24T00:05:00+02:00

This was my first Zend Conference, and I must say I had a great time there. The organization was top notch, there were plenty of good sessions, and the unconference topics proved to be very interesting.

I've also met and talked with a lot of great people. I have particularly enjoyed the talks I had with the Zend team, and with all the people I had email conversations with in the past but never had the chance to meet in person.

Beside my talk about how to use symfony within a Zend Framework project, I have also held an uncon session about Twig, and took part in a roundtable about Dependency Injection.

If you have not attended Zend Con this year, most of the presentations are now available online on both slideshare and joind.

Last but not the least, I have been invited yesterday by the San Francisco symfony user group to talk about Symfony 2, and I had a great time talking with SF symfony users.

Thanks again Zend for inviting me this year, and I hope to come back next year.

Templating engines in PHP - Follow-Up

2009-10-09T15:10:00+02:00

My post about template engines in PHP received more than 70 comments as of now, and counting. That's a lot considering most of them are really well thought out, and backed with solid counter-arguments. Thanks everybody for taking the time to participate constructively to the discussion. I'm really proud that the PHP community (or at least the small part which reads my blog) is able to discuss such a touchy topic without immediately starting a flame war! I'm also impressed about how many people need to promote their own template engines ;)

Before I try to answer some questions, I'd like to reinstate that I like PHP templates. And you should remember that symfony has only used plain old PHP templates since the beginning. As a matter of fact, I'm been advocating about using PHP templates since my first PHP project, and I have never used any other PHP template engines. So, I'm not against PHP templates; I just find that some PHP limitations as a template language are more and more irritating for me.

And as Eli has pointed out, "[I probably have] grossly undersold PHP-as-template in [my] enthusiasm to promote [my] new templating language that [I] created".

I also like Tchalvak comment: "The question of PHP vs a templating engine isn't the essential one, the essential (and easily answerable one) is templates vs. no templates. Templates - and the separation of display vs. application logic that they bring with them - are a necessity. What form they come in is much less important.". He sums up really well the question. More on that later on.

I understand all the points raised in the comments, and I basically agree with most of them. Now, let me answer some of the more interesting questions.

Twig Background

I started to look for a template engine a few months ago. People who know me also know that I don't like to reinvent the wheel. So, I didn't want to create a new library from scratch.

I looked for a good template engines, I tried some of them, and finally found Twig. But as soon as I started to use it, I found that it was exactly what the gem I was looking for (because of the features it already had and also because of its clean and beautiful architecture). As Twig was not a standalone project, but rather an embedded project, I started to hack it a bit, and after I made several enhancements like the sandboxing feature, I decided to contact Armin to discuss the future of Twig.

So, you are not sold... yet?

Even if you don't buy my arguments, that's fine. I don't want Twig to become the universal template language for PHP, far from it. I really think there is a market for template engines like Twig, but I'm the first to recognize that it won't be used for all PHP projects to come!

If you are looking for a templating engine that only uses PHP and have built-in support for template inheritance, blocks, helpers, and some more, the good news is that I have coded one some weeks ago under the Symfony Templating Component name. It's a standalone component, which has no other dependencies, and I'm sure most people wanting to use plain PHP for their projects will love this project.

And better yet, you can use both Twig and the Symfony Templating component to have the best of both worlds. Use the component for all your templates and Twig if you need the sandboxing feature.

About the Syntax

Lots of comments about the syntax, and the problem I raised about the PHP syntax. It's not about having a prettier syntax and if you think that the syntax issues I mentioned are mostly bikeshed color arguments, think again.

One of the key point of my reasoning is that a good template language should aim to find the sweet spot. The template language should find the best compromise about too many and not enough features. As I said in my previous post, the template language is all about presentation logic. And of course, simple conditions and loops are part of the presentation logic. But do you want to use array_chunk() in a template? Probably not. This call belongs to the controller or the model, depending on what you want to do.

Also, templates are about a lot of HTML with some PHP. So, this kind of code snippet is a big no-no for me in a template:

<?php
    if ($items) {
        foreach ($items as $item) {
            echo "* {$item}\n";
        }
    } else {
        echo "No item has been found.\n";
    }
?>

Many people seem to like the PHP short tags. First, about the math. If you compare <?= $var ?> with <?php echo $var ?>, the difference is 7 characters, not 2. But that's really not the main problem.

Apart from problems like XML support, the short_open_tag setting, the shared hosts configuration issue, and some more, it's also about coding standards:

PEAR:

"Always use <?php ?> to delimit PHP code, not the <? ?> shorthand. This is required for PEAR compliance and is also the most portable way to include PHP code on differing operating systems and setups."
Zend:

"Short tags are never allowed."

The Pear and Zend projects are serious, so there should have some reasons to disallow short tags, no?

But as Eli mentioned in his post, I would also like to see a PHP evolution to take this problem into account: "... there are a number of people (including myself [Eli]), who have been tossing around the idea of proposing a new option to the short_tags directive for PHP, allowing not just having them turned on or off. But allowing a 3rd option, that would enable <?= ?> while disabling <? ?>"

About Web Designers

The debate is not about if web designers should understand PHP or not. And it's certainly not about web designers not being smart enough. Of course web designers can learn a bit of PHP... until they learn too much and start getting stuff that do not belong to the template in their templates (like getting records directly from the database, anyone?) Oh, of course, they won't do such a mistake it they have also learned about the MVC pattern. But then, they are not web designers anymore, they are web developers.

Finding this good balance is all about providing a tool that tries to help web designers not to shoot themselves in the foot. And I think everybody will be with me if I say that PHP does a poor job setting the limits.

Output Escaping

Some commenters advocate that output escaping should be done in the controllers. That cannot work. As John Campbell writes: "The problem is that the controller can't possibly be aware of the context of the output, nor understand which type of escaping is correct."

The issue is therefore not that simple. On the one hand, developers should take care of output escaping, but it cannot be done in controllers. On the other hand, web designers should not have to take care about output escaping, but templates is the only place where you have enough context information to apply sensible escaping. That's why automatic output escaping seems the best compromise to me, plus the fact that being "almost" secure by default is a big advantage.

Speaking of automatic escaping, I'm probably someone who knows a lot about it as symfony has this feature since early 2006. I can tell you that the performance overhead of such a feature is very high. In Twig, it is added during compilation, so there is no overhead whatsoever.

About Sandboxing

The sandbox feature is not targeted at web designers. It is mainly useful for situation where external people can change the templates (think of webmasters able to configure their CMS or blogging platform templates from a backend interface).

The Dwoo and Smarty security feature is a right step in that direction, but not as powerful as a full sandbox feature. As far as I understand, it's mainly about allowing PHP code or not in the templates, and restricting the PHP functions you can use (of course, correct me if I'm wrong).

Sandboxing controls everything, from the tags you can use, to the methods you can call on the objects (the Twig documentation explains the concept and how it works).

About Speed

Most template engines compile the templates down to PHP code. So, the speed of lexing, parsing, and compiling is not that relevant. What matters is the speed of the evaluation. Here is the compiled version of the Hello {{ name }} template:

/* Hello {{ name }} */
class __TwigTemplate_1121b6f109fe93ebe8c6e22e3712bceb extends Twig_Template
{
  public function display($context)
  {
    $this->env->initRuntime();
 
    // line 1
    echo "Hello ";
    echo (isset($context['name']) ? $context['name'] : null);
  }
}

Some people were worried about debugging. As you can see, the generated code is very lean, and contains the template name and the lines of the original template. That should be more than enough to debug all the possible problems easily.

You can find it much more verbose than the PHP version, and of course, for such a contrived example, the native PHP version is insanely simpler:

Hello <?php echo $name ?>

But have a look at the generated code for many template engines and you will see for yourself that Twig output is one of the cleanest and shortest one.

As asked by many people, I have redone my benchmark by bypassing the compilation phase altogether (the cache has been primed before launching the benchmark scripts). As I expected, it does not change numbers significantly. The compilation of such simple templates is insignificant compared to the rendering of 10,000 compiled templates.

I have also tested plain PHP templates. So, here is the updated table:

Library	Time (sec)	Memory (Kb)
Plain PHP	2.4	114
Twig	3	383
PHPTAL	3.8	598
Dwoo	6.9	1,645
Smarty 2	12.9	610*
Smarty 3	14.9	799*
Calypso	34.3	614
eZ Templates	53	2,783

Notice that the memory used by Smarty is much lower than the memory used in my previous benchmark as I wrapped the whole loop with an ob_start()/ob_end_clean(), which was not fair comparing to the other template engines. This was fixed in this new benchmark.

So, as expected, PHP is faster than Twig for simple templates. But as templates become more complex, the difference will be less and less noticeable, due to the clean code generated by Twig.

The benchmark scripts are executed on the command line without any PHP accelerator. A PHP accelerator in this case does not help as everything is done in one single PHP process.

You can download the benchmark code here: http://fabien.potencier.org/benchmarks.tgz

Twig and Symfony

This section is for people worried about the integration of Twig as the default template engine in symfony.

symfony 1.3/1.4

Obvisouly, Twig won't be part of symfony 1.3 as the feature-freeze deadline is the end of next week and also because we want the symfony 1.3 release to be an evolution of symfony 1.2, and not a revolution.

If some people want to create a Twig plugin for symfony 1.3, start the discussion on the symfony developer mailing-list, and I will help you getting started.

Symfony 2.0

As far as Symfony 2 is concerned, nothing is definitive yet, but Twig won't be the default template language. I think Twig will be available as an optional plugin, well integrated with the core. For instance, it will probably make sense to use it for the admin generator to provide some specific tags that will ease the customization of it.

And thanks to the Symfony Templating Component, developers will be able to mix PHP templates and Twig templates easily in one project, depending on their needs or the tools they want to use.

So, if you can't stand template engines, use plain PHP templates; but if you want to benefit from Twig, use it natively. And of course, if you use plugins that made the opposite choice, that won't be a problem either.

Conclusion

Hopefully, this new post has answered some of the points raised in the comments.

Templating Engines in PHP

2009-10-07T23:40:00+02:00

This blog post is not for the faint-hearted! Some people will strongly disagree with me and some others will probably want to kill me at the upcoming Zend Conference. And if starting an argument in the comments can help you feel better, please feel free to do so. If you want to have a more advanced discussion on this topic, vote for my talk at the Zend UnConference.

So, you think PHP is a templating engine? So did I... for a very long time. But recently, I changed my mind. Even if PHP can be used as a templating engine, the syntax is just plain ugly as a template language.

For several years now, I have been promoting web development best practices, and one of them is the separation of concerns. Of course, as the lead developer of symfony, all the projects we work on at Sensio are modeled after the MVC architecture. It certainly helps when we have big projects where many people need to work together. The developers work on the code (the Controllers and the Model) and the web designers work on the design. And templates are sometimes written by developers, but more often than not, they need to be written by web designers or by the webmasters themselves.

And a template language is something that helps you to write templates that respects this separation of concerns. A template language should find a good balance between giving enough features to ease implementing the presentation logic, and restricting the advanced features to avoid the business logic to cripple your templates.

So, when I asked a few days ago about the best and popular templating engines in PHP on Twitter, some people naturally answered "PHP" itself. I was not even surprised as that would probably have been my answer some weeks ago too.

Why PHP is not (anymore) a good template language?

Why do people still think PHP is a templating engine? Sure enough, PHP started its life as a template language, but it did not evolve like one in the recent years. If you think PHP is still a template language, can you give me just one recent change in the PHP language which enhanced PHP as a template language? I cannot think of one.

Template languages evolved a lot since 1995 and the initial release of PHP/FI:

<!--include /text/header.html-->
 
<!--getenv HTTP_USER_AGENT-->
<!--ifsubstr $exec_result Mozilla-->
  Hey, you are using Netscape!<p>
<!--endif-->
 
<!--sql database select * from table where user='$username'-->
<!--ifless $numentries 1-->
  Sorry, that record does not exist<p>
<!--endif exit-->
  Welcome <!--$user-->!<p>
  You have <!--$index:0--> credits left in your account.<p>
 
<!--include /text/footer.html-->

And as a matter of fact, PHP doesn't support many features modern template languages should have nowadays.

I will take Django as an example of a modern template language in my examples for reasons you will understand later on, and mainly because I think Django template language hits that sweet spot I talked about above.

The following sections describes the main features I want to find in a modern template language:

Concision

The PHP language is verbose. You need no less than 14 characters just to output a simple variable (and no, using the more compact <?= shortcut is not an option):

<?php echo $var ?>

And PHP becomes ridiculously verbose when it comes to output escaping (and yes, escaping variables coming from an unsafe source is mandatory nowadays):

<?php echo htmlspecialchars($var, ENT_QUOTES, 'UTF-8') ?>

Compare with the same examples written with the Django template language:

{{ var }}
 
{{ var|escape }}

Template oriented syntax

This one is mostly a matter of taste, but modern template language have nice idioms to express common needs. For instance, let's say you want to iterate over an array and want to display a default text when the array is empty. That's very common, but the PHP version is not very readable:

<?php if ($items): ?>
  <?php foreach ($items as $item): ?>
    * <?php echo $item ?>
  <?php endforeach; ?>
<?php else: ?>
    No item has been found.
<?php endif; ?>

The Django version is much better thanks to the use of an else clause for the for tag:

{% for item in items %}
  * {{ item }}
{% else %}
  No item has been found.
{% endfor %}

Reusability

PHP has greatly evolved over the years as far as reusability is concerned. Since PHP5, the object implementation is much better, and when traits will be supported in the next version of the language, we will have a solid general purpose language at our disposal. I'm really happy with all these changes as it helps me write better programs, but these enhancements are irrelevant when what you want to do is to mainly write templates.

Django introduced template inheritance some years ago, as a way to mimic classes but for templates:

<!-- base.html -->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
  <head>
    {% block head %}
      <link rel="stylesheet" href="main.css" />
    {% endblock %}
  </head>
  <body>
    {% block content %}{% endblock %}
  </body>
</html>
 
<!-- index.html -->
{% extends "base.html" %}
 
{% block head %}
  {{ block.super }}
  <link rel="stylesheet" href="main.css" />
{% endblock %}
 
{% block content %}
  Index content
{% endblock %}

It's elegant, easy to understand, and really powerful. It's so powerful that many template engines now support this feature out of the box.

Security

I'm not saying PHP is not a secure language, far from it. But needless to say that escaping a variable in a template is just a nightmare as I have showed you previously:

<?php echo htmlspecialchars($var, ENT_QUOTES, 'UTF-8') ?>

Of course, you can create your own function to make it shorter, but that's not my point:

<?php echo e($var) ?>

For me, security should be enabled by default, especially for templates written by non-developers who are not necessarily aware of the common web threats like XSS or CSRF.

As far as I know, symfony was one of the very first web frameworks to have automatic output escaping for variables used in templates (2006); and in the recent years, major frameworks followed the same path: Django has automatic output escaping enabled since 1.0, and Ruby on Rails will also have it in the upcoming version 3.

Having output escaping enabled by default also means that auditing an application is much easier. Just have a look at templates where escaping has been disabled:

{% autoescape off %}
  {{ object.as_html }}
{% endautoescape %}
 
{{ object.as_html|safe }}

Of course, I'm well aware of the automatic output escaping problems. You still need to take care of escaping variables in JavaScript tags correctly, but that's much easier to remember than just having to escape everything by hand.

Sandbox mode

This one is a must-have when you allow users to edit templates (when webmasters are allowed to edit some templates from a web backend for instance). That's not a universal need, but one that needs to be addressed anyway. Evaluating a template in a sandbox means being able to restrict what can be done in it. For instance, you should be able to restrict the methods/functions that can be called, the tags that can be used, ...

Neither PHP nor Django have a sandbox mode, so keep reading this article to learn more about this topic.

Alternative template languages in PHP

So, I started looking for a template engine that has all the features I mentioned above. I found many different template engines but none satisfied all my needs. The next sections talk about some of them and some others people suggested in reply to my tweet.

As PHP have a million template engines, I have only tested and benchmarked the more "popular" ones. And as I don't use these libraries, the following sections can contain wrong information. In such a case, please correct me in the comments and I will then fix this post accordingly.

Smarty and Smarty 3

That's the first one that comes to mind. And Django template language itself was inspired by Smarty. Smarty is the de-facto PHP standard template engine.

Hello {$name|escape}
 
{section name=item loop=$items}
  {$items[item]|escape}
{/section}

But Smarty suffers from several problems:

Not object oriented
No template inheritance
No sandbox mode
No automatic escaping

As far I understand, Smarty 3 is just around the corner and will improve the library quite a lot:

Object oriented architecture
Auto escaping of variables
Template inheritance

I tested the two versions, but the performance of both are quite bad (see at the end of this post for more information on the benchmark I did).

PHPTAL

PHPTAL is a very nice project that implements the Zope Page Templates syntax. It is very well designed, supports lots of features, but is unable to render templates besides HTML ones, which can be a problem if you want to use the same language for emails, RSS feeds, and so on.

<?xml version="1.0"?>
<html>
  <body>
    Hello <span tal:content="name" />
    <ul tal:repeat="item items">
      <li tal:content="item"></li>
    </ul>
  </body>
</html>

Also, I think the syntax is not web designer friendly, especially when you use advanced features like template inheritance:

<html metal:use-macro="layout.xml/main">
 <body metal:fill-slot="content">
   Hello <span tal:content="name" />
   <ul tal:repeat="a array">
     <li tal:content="a"></li>
   </ul>
 </body>
</html>

<html metal:define-macro="main">
  <metal:block define-slot="content"/>
</html>

One of the biggest selling point of PHPTAL is the autocompletion you can have in IDEs, and a guarantee that your HTML is well-formed.

eZ Components Templates

eZ Components Template Component is also a very nice implementation of a template language. It's probably the one which have the most features... perhaps too many of them if you ask me:

{use $name}
{use $items}
 
Hello {$name}
 
{foreach $items as $item}
 * {$item}
{/foreach}

It does not support template inheritance, and my main concern is the performance. This it is the slowest library I have tested, and by great margin.

Dwoo

Dwoo is an interesting project. It is positioned as an alternative to Smarty. And they did a great job:

<html>
  <body>
    {block "content"}{/block}
  </body>
</html>

{extends "layout.tpl"}
 
{block "content"}
  {include("basic.tpl")}
{/block}

Dwoo mimics Smarty but with some interesting new features like template inheritance, and with much better performance than Smarty.

Unfortunately, Dwoo has no sandbox feature and its core is not flexible enough.

Calypso

Calypso is an implementation of the Django Template Language in PHP. I mention it because it's a clone of Django and because some people mentioned it on Twitter. But, the author himself acknowledge that the implementation is flawed.

Twig

When I started looking for a PHP template language, I focused on libraries trying to mimic the Django template language features. After many hours of Googling, I found Twig. Twig was written by Armin Ronacher of Jinja fame. Needless to say I have the uttermost respect for Armin as he does a wonderful job with Jinja. As a matter of fact, Twig is more similar to Jinja than Django as far as the implementation is concerned.

He wrote Twig back in 2008 for Chypr, a blogging platform. But he never really developed it further as he mostly work with Python.

When I had a look at the code, I was immediately sure it was what I was looking for. The main difference with Calypso being the fact that Twig compiles the templates down to plain PHP code. I started to use it a bit and at the end of last week, I asked Armin if he wouldn't mind letting the project starts a new life. His answer was enthusiastic, and so I began hacking the code. My version is much different from that of Armin, but the lexer and parser are mostly the original ones.

I have just hacked the code for a few days, but I'm already quite proud of it, and I think it's time to open it to the public. So, yesterday I wrote some documentation for it and put up a simple website. There is still a lot of work to do like finishing the documentation, adding more unit tests, and PHPdoc, but the code is already quite solid and feature-full with:

Native template inheritance (templates are compiled as classes);
Solid automatic auto-escaping (with no associated runtime overhead as everything is done during compilation);
Very secure sandbox mode (white-list the tags, filters, and methods that can be used in templates);
Great extensibility: you override everything, even the core features, by bundling your own tags and filters as an extension; but you can also manipulate the AST (Abstract Syntax Tree) before compilation. By leveraging this possibilities, you can even create your own DSL (Domain Specific Language), targeted at your application.

Despite being one the most full-featured PHP templating engines, Twig is also the fastest one:

Library	Time (sec)	Memory (Ko)	Templates rendered per second
Twig	3	1,190	3,333
PHPTAL	3.8	2,100	2,632
Dwoo	6.9	1,870	1,449
Smarty 2	12.9	2,350	775
Smarty 3	14.9	3,230	671
Calypso	34.3	620	292
eZ Templates	53	5,850	189

I benchmarked a template that includes a simple template (one output and a for loop on three items) and decorated with a simple layout. The times are for the average of ten runs; a run consisting of one compilation of the template and 10,000 rendering of it. For engines that do not support inheritance, I have used a header and a footer instead, and for engines that do not support automatic output escaping, escaping has been done by hand.

When the templates are already compiled, the memory consumption is of course much better for all template engines, and Twig is the one which uses the less memory:

Library	Memory without compilation (Ko)
Twig	383
PHPTAL	598
Dwoo	1,645
Smarty 2	1,634
Smarty 3	1,790
Calypso	614
eZ Templates	2,783

If you want to learn more, please visit the Twig website, and start discussing Twig on its dedicated mailing-list. If you want to join the team, subscribe to the developer mailing-list

I have also planned to talk about Twig at the Zend UnConference. Vote for this talk and see you there for an interesting discussion about templating engines.

Swift Mailer Takeover

2009-09-16T15:30:00+02:00

As you might have noticed if you follow me on Twitter, I'm the new project manager of Swift Mailer, a powerful mailing library for PHP.

This is a great honor for me to take over such a project. The Swift Mailer project has been initiated by Chris Corbyn some years ago. He has done a wonderful job during the years, trying to find the best way to create a top-notch emailing solution for PHP. With the release of the 4th version of Swift Mailer earlier this year, I think we now have a very solid mailing library. It is one of the most beautiful code I have had the pleasure to read. Also, the library is feature-rich, object-oriented, and very modular thanks to its plugin architecture. The good news is that Chris will still work on the project, so the takeover will be smooth.

Unfortunately, it seems that Swift Mailer has not been as widespread as it deserves to be. One of my goal is to spread the word about it, and of course, if you are looking for a good solution to send emails, try it, you won't be disappointed. It is already used by companies sending ten of thousands of emails per hour!

As a good news never comes alone, as we say in France, Swift Mailer 4 will also be supported as the default emailing solution as of symfony 1.3.

I also have lots of ideas to move the project forward. One of them is to better support the intricacies of Japanese emails. Another one is to have better documentation, and more people talking about the project.

Of course I need help. So, if you want to join a great Open-Source project, visit the Swift Mailer website today, subscribe to the mailing-list, and help us build a great PHP library!

Developers should be Artists

2009-08-25T16:41:00+02:00

During my holidays, I took the chance to step back a little and think about my day-to-day work. As the CEO of Sensio, I have the opportunity to do a lot of different jobs. But what I really like to do is hacking. By hacking, I mean developing software for the fun of it, software that is not for a "paying" customer. And as such, hacking cannot be a day job per se. Hacking is something you do on the side. It's a hobby, a passion. And Open-Source is probably the best way to share your hacks.

Why do I like hacking so much? Obviously because I'm free to do whatever I want, with the tools and the methodology I choose. But more important, because I have the freedom to do it iteratively with no schedule pressure. Building a software is a long iterative process. I love to be able to code something, forget about it for some time, and come back to it later on to enhance it, or throw it away altogether. But back to the artist side of things. How much time do you think Leonardo da Vinci took to paint the Mona Lisa? According to Wikipedia, he began painting the Mona Lisa in 1503 and he is thought to have finished it shortly before he died in 1519. What? 13 years for just one painting? That's insane. Perhaps, but that's the way it works for artists. I started developing symfony in 2004, and it still unfinished in 2009, and probably will never be (and the number of features has not raised that much in five years). Hacking is an art, and hackers should act as artists. Hackers and painters have a lot in common but I won't talk about the analogy too much as Paul Graham wrote an excellent essay and a whole book on this topic. I recommend you to read both of them if you are a hacker or a developer.

Hacking is about creating something by using the computer as a medium of expression. One of the best hacker and artist I know of is Why the lucky stiff. From the beautiful camping micro-framework, to the excellent "Poignant Guide to Ruby" book, he was very prolific and a real artist. I use the past tense, as he recently removed all his online presence.

Everyday, I try to learn new ways to write beautiful code in PHP (who's laughing at me?). You can easily follow my progress by watching the evolutions of the symfony code, and more recently the symfony components code.

If you are looking for more "programming art", I also did some attempts with Twitto and Twittee, my two Twitter experiments.

// Twitto
require __DIR__.'/c.php';
if (!is_callable($c = @$_GET['c'] ?: function() { echo 'Woah!'; }))
  throw new Exception('Error');
$c();

// Twittee
class Container {
 protected $s=array();
 function __set($k, $c) { $this->s[$k]=$c; }
 function __get($k) { return $this->s[$k]($this); }
}

Back in the early nineties, I did some similar experiments. At that time, the goal was to do as much as possible in 10 lines of code. I was once published in the French ST Magazine (August 1990) for a program done is STOS (you can see the wonderful program in the sidebar).

If you ask me, a hacker should definitely be an artist. But can a developer working for a company be an artist? Unfortunately, I don't think so. To be an artist, you need time and freedom, two essential things you rarely have when working for a customer.