Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In with Google Sign In with OpenID

Experiment: Autoloader testing, class map vs default autoloader

edited February 2012 in Framework

As per the suggestion here and this blog post on autoloader benchmarking, I got to experimenting with Elefant's autoloader to see if I could make it faster.

Note: All code for this experiment has been posted here:

https://gist.github.com/1718897

First, I needed to determine which caching method would be fasted, using json_encode(), serialize(), or generating raw PHP you could include(). To see which was faster, I generated 10,000 files of each type based on a real map of Elefant's classes. Then I ran benchmarks against the three techniques first by parsing all 10,000 files then simply parsing the same file 10,000 times, both with APC enabled and disabled. Here are the raw results, showing unserialize() as the clear winner whether APC is enabled or not:

Reading and parsing 10,000 separate files

APC Format Duration Memory
Off JSON 0.59806108474731 677056
Off JSON 0.61035680770874 677056
Off JSON 0.62755990028381 677056
Off JSON 0.59243798255920 677056
Off Serialize 0.46707820892334 677408
Off Serialize 0.43602800369263 677408
Off Serialize 0.43495607376099 677408
Off Serialize 0.45977807044983 677408
Off PHP 0.72282910346985 6233968
Off PHP 0.68916916847229 6643320
Off PHP 0.71095204353333 6643320
Off PHP 0.67504096031189 6643320
On JSON 0.61713099479675 677056
On JSON 0.60398292541504 677056
On JSON 0.59711790084839 677056
On JSON 0.57904291152954 677056
On Serialize 0.43216991424561 677408
On Serialize 0.41977500915527 677408
On Serialize 0.41345000267029 677408
On Serialize 0.42559218406677 677408
On PHP 0.66842508316040 6643320
On PHP 0.65937495231628 6643320
On PHP 0.67941093444824 6643320
On PHP 0.69633197784424 6643320

Reading and parsing the same file 10,000 times:

APC Format Duration Memory
Off JSON 0.55757498741150 677576
Off JSON 0.55717897415161 677576
Off JSON 0.56575489044189 677576
Off JSON 0.56445479393005 677576
Off Serialize 0.37442612648010 676712
Off Serialize 0.37197208404541 676712
Off Serialize 0.37523198127747 676712
Off Serialize 0.38755488395691 676712
Off PHP 0.58293008804321 672752
Off PHP 0.60621094703674 672752
Off PHP 0.59600687026978 672752
Off PHP 0.59908008575439 672752
On JSON 0.54677200317383 677576
On JSON 0.54175305366516 677576
On JSON 0.54964685440063 677576
On JSON 0.56942200660706 677576
On Serialize 0.37199902534485 676712
On Serialize 0.38216996192932 676712
On Serialize 0.37098908424377 676712
On Serialize 0.37338995933533 676712
On PHP 0.57700991630554 672752
On PHP 0.58972120285034 672752
On PHP 0.59540891647339 672752
On PHP 0.59390807151794 672752

So now I could write the new autoloader using unserialize() for the class map parsing:

https://gist.github.com/1718897#file_autoloader.php

I also wrote some quick unit tests to make sure the mapping worked correctly:

https://gist.github.com/1718897#file_autoloader_test.php

To benchmark these, I wrote a quick script that generated 10,000 classes and then autoloaded them all using the old and new autoloaders. Here were the initial results:

APC Autoloader Duration Memory
Off Old 0.63187599182129 19981536
Off Old 0.63982796669006 19981536
Off Old 0.6190390586853 19981536
Off Old 0.63123202323914 19981536
Off New 0.62571692466736 22938688
Off New 0.62852692604065 22938688
Off New 0.65477299690247 22938688
Off New 0.6645450592041 22938688
On Old 0.69804811477661 19981536
On Old 0.65681409835815 19981536
On Old 0.67953777313232 19981536
On Old 0.63812994956970 19981536
On New 0.68386101722717 22938688
On New 0.64614295959473 22938688
On New 0.65268611907959 22938688
On New 0.66691088676453 22938688

From these it appears that the only real difference is that the class mapped version took additional memory, but wasn't yielding any real performance gains. I tested the two against Elefant itself using ab -n 1000 -c 100 on the homepage of a default Elefant 1.1.2 install, and found that the performance was almost identical in practice. Checking the memory usage, it went from 1.7MB/request to 1.8MB with the class map.

So at this point, unless there are some additional optimizations that I missed (which I would love to know about :), it looks like Elefant's existing autoloader is working just fine. Additionally, the class map has the downside that if you move a class file (but don't rename it), the autoloader would look for it in the old place unless you rebuild the class map. This could add needless confusion for developers on the rare occasion.

Note: I left out results from experiments with file paths that the previously mentioned blog post included, since they didn't show a noticeable difference in my own tests (with and without APC).

Comments

  • This classmap version write cache immediately. Writing cache at the end seems better.

  • It only updates the file when a new class is added to the map, so usually only the first page load or two will write to disk and then after that it'll only be reading from the cache. This makes it automatic but still fast, without worrying about running a pre-generator script.

  • I used to see some frameworks have a class map file in each module (or plugin), I think it is used to improve performance, as it can reduce disk IO request. Thanks to your work!

    My comment above means, do not need to update cache immediately. When the script shutdown, update the cache, this will be better. It is a small improve, doesn't change the result above.

  • Once it writes the class to the cache, it never has to do so again, so only the first request is penalized. This works just as fast, but keeps it simpler.

    I've tweaked the test again a bit and included the full file paths in the class map, and it looks like I'm getting a very minor speed up now (at the expense of a little extra memory usage):

    APC Autoloader Duration Memory
    Off New 0.53757095336914 23428072
    Off New 0.53285288810730 23428072
    Off New 0.54046392440796 23428072
    Off New 0.56175804138184 23428072
    On New 0.53356814384460 23428072
    On New 0.54588294029236 23428072
    On New 0.54997992515564 23428072
    On New 0.55416488647461 23428072

    (compared to the last table in my previous results)

    However, in practice testing against the real Elefant codebase, individual page requests as well as requests/second came out almost completely even across the board.

    Since adding the full paths also adds potential confusion when developers move their sites from one machine to another, or even one folder to another, the negligible gains don't seem to be worth it. I think we can now safely say Elefant's existing class loader is good enough :)

  • I agree :)

Sign In or Register to comment.