Elefant beats Lumen on performance

jbroadway · April 2015

I just saw the announcement of the new Lumen PHP micro-framework on Hacker News, and I saw that they claim to be substantially faster than both Silex and Slim 3, so I thought I'd do a quick benchmark to see how Elefant stacks up.

Keeping in mind that Elefant is a full blown CMS on top of a framework, I figured maybe this time my luck would run out.

For my setup, I'm running Nginx + PHP 5.5.9 on Ubuntu 14.04 through Vagrant + Virtualbox on a 2014 MacBook Pro. Here's my Elefant handler, saved to apps/hello/handlers/index.php:

<?php

$page->layout = false;

list ($user) = $this->params;

echo 'Hello ' . $user;

And here is my Lumen route, saved to app/Http/routes.php:

<?php

$app->get ('/hello/{user}', function ($user) {
    return 'Hello ' . $user;
});

Doing a quick ab test of each, I'm consistently getting results above 400 req/s on Elefant, and I haven't hit 300 req/s once on Lumen:

Requests per second:    494.12 [#/sec] (mean)
Time per request:       20.238 [ms] (mean)
Time per request:       2.024 [ms] (mean, across all concurrent requests)
Transfer rate:          84.93 [Kbytes/sec] received

Versus:

Requests per second:    278.37 [#/sec] (mean)
Time per request:       35.923 [ms] (mean)
Time per request:       3.592 [ms] (mean, across all concurrent requests)
Transfer rate:          47.57 [Kbytes/sec] received

At higher concurrency levels (100 vs 10), Elefant still maintains a roughly 25 req/s lead over Lumen:

Requests per second:    165.92 [#/sec] (mean)

Versus:

Requests per second:    139.88 [#/sec] (mean)

This makes me very happy, since my work lately has involved a lot of microservice development, and having the full Elefant framework at my disposal has been super handy, and I love that I'm not making a performance sacrifice for it.

Techborn · April 2015

That's really impressive. Ya know, another benchmark that I'd be interested in is comparing Elefant to the Phalcon framework (http://www.phalconphp.com/). I've heard many good things about that framework and it's speed from a couple of my php friends, and I'd be interested in how elefantCMS compares to it.

jbroadway · April 2015

I would assume Phalcon would win out, being a PHP extension written in C. But you're right, I should test and see how it stacks up! :)

jbroadway · April 2015

I started toying with writing some of Elefant's core classes in Zephir, which is the new language Phalcon is being rewritten in I believe, but I got side-tracked as usual ;)

jbroadway · April 2015

Quick test with Phalcon gives me between 900-1200 req/s versus the scores above.

I should test with HHVM and PHP 7 now too haha

jbroadway · April 2015

Okay, did a bit more Phalcon testing. My original Phalcon test was using its "micro" version, comparing Elefant against this code:

<?php

$app = new Phalcon\Mvc\Micro ();

$app->get ('/hello/{user}', function ($user) {
    echo 'Hello ' . $user;
});

$app->handle ();

To get a better sense of the Phalcon framework, I split it into two files as per Phalcon's documentation:

The front controller:

<?php

try {
    $loader = new \Phalcon\Loader ();

    $loader->registerDirs (array (__DIR__ . '/app/controllers/'))->register ();

    $di = new Phalcon\DI\FactoryDefault ();

    $di->set ('view', function () {
        $view = new \Phalcon\Mvc\View ();
        $view->setViewsDir (__DIR__ . '/app/views/');
        return $view;
    });

    $app = new Phalcon\Mvc\Application ();

    $app->setDI ($di);

    echo $app->handle ()->getContent ();
} catch (Phalcon_Exception $e) {
    echo 'PhalconException: ', $e->getMessage (), PHP_EOL;
}

And the request handler:

<?php

class IndexController extends \Phalcon\Mvc\Controller {
    public function helloAction ($user) {
        echo 'Hello ' . $user;
    }
}

This reduced Phalcon's performance to:

Requests per second:    601.52 [#/sec] (mean)
Time per request:       16.625 [ms] (mean)
Time per request:       1.662 [ms] (mean, across all concurrent requests)
Transfer rate:          102.80 [Kbytes/sec] received

Still a big jump up from Elefant's 494.12 req/s, but not so far off now. Impressive that a C-based framework doesn't beat a pure PHP framework by that much once you add in the setup for a real world app structure.

With both PHP 7 and HHVM working to level those differences even more, PHP really is getting pretty fast! :)

Howdy, Stranger!

Categories

Elefant beats Lumen on performance

Comments