Benchmarking Autoloading vs Combining classes into a single file

Thinking about ways of further improving PHPixie I started looking at other projects for inspiration. For example the Fat-Free framework boasts on being contained in a single file. This got me thinking about making a tool for merging all project classes together with vendor libraries into a single for performance boost. MAking such a tool is a fairly trivial task, but still I wanted to be sure it would actually be usefull, so I decided to benchmark autoloading classes with composer vs combining them into a single file.

The setup was fairly simple:
1) Generate a set number of classes
2) Build a composer classmap for them to speed up autoloading
3) Combine them into a merged file
4) Benchmark the amount of time spent instantiating the classes
5) Repeat steps 1-4 for a number of classes ranging from 1 to 500

To generate classes that would simulate realistic IO each class would get a random number of methods with random number of lines in them. Finally the class generator would look like this:

function build_method($class,$i) {
  $str = "    function {$class}_{$i}() {\r\n";

        for ($i = 0;$i<rand(15,30);$i++){
                $str.= '        $i = '.rand(100, 200).' + '.rand(10, 90).";\r\n";
        }

        $str.= "}\r\n";
        return $str;
}

function build_header() {
        $str="<?php\r\n";
        $str.= "namespace A;\r\n";
        return $str;
}
function build_class($i){

        $extends = $i%10 > 0?"extends A".($i-1)." ":"";

        $str= "class A$i $extends{\r\n";
        for($j = 0; $j < rand(4, 9); $j++)
                $str.= build_method("A$i", $j);
        $str.= "}\r\n";
        return $str;
}

$merged = "";
for($i=0;$i<$argv[1];$i++){
        $class = build_class($i);
        $merged.=$class;
        file_put_contents(__DIR__."/A/A{$i}.php", build_header().$class);
}

file_put_contents(__DIR__."/merged.php", build_header().$merged);

To add even more realism I made it so that some classes are children of other classes, this way instead of instantiating every class to load it, I can just instantiate classes A9 and A19 to load all classes from A0 to A19 in 2 calls. From here it was just a matter of setting up 2 php, files, one that would execute testcases using autoloaded clasess, the other using the merged ones.

So here are the results:

Benchmarking loading of all generated classes for each test

Benchmarking loading of all generated classes for each test

As wee can see on the first chart, with XCache disabled the Combined approach outperforms Autoloading as the number of classes increases, which is quite logical, the performance difference is about 0.03ms with 500 classes which is really negligible. Now let’s try turning XCache on:

Loading classes with opcache enabled

Loading classes with opcache enabled

Xcache pretty much levels everything by caching the opcodes, thus negating any reason to combine classes into a single file. I’ll go even further and prove that combining classes into a single file will actually hurt performance. The reason being is that in both tests we loaded all of the project classes, this doesn’t happen in real life though.

If you have a large project that has lots of different logic in it I’d say that on each request only about 10% of all project libraries are needed. In this case autoloading will work much faster, since it doesn’t load useless code and hog memory like the combined approach would.

So here is my final benchmark. For this one I load only 10% of generated classes (so for 500 generated classes in a test only 50 are being used):

Loading only 10 percent of generated classes

Loading only 10 percent of generated classes

As we can see Autoloading dramatically outperforms the merged file approach. Now I don’t mean to say that Fat-Free isn’t a fast framework, I just think that this particular aspect of it isn’t really usefull. So I guess PHPixie will not be getting a “combine your classes into a single file” tool after all.

So is there anything that we still can do to improve autoloading performance? Yes! We can use composer properly.

$loader = require 'vendor/autoload.php';

//This is the WRONG approach
//because adding your files like this
//you won't benefit from the classmap cache
$loader->add('Acme\\Test\\', __DIR__);

The correct approach is to put this in your composer.json:

{
	"autoload": {
		"psr-0": {"": "classes/"}
	},
}

//Now if you run "php composer.phar update -o"
//A classmap cache will include your project files too
//thus (slightly) boosting autoload speed

3 thoughts on “Benchmarking Autoloading vs Combining classes into a single file

  1. ikkez

    Hi dracony.
    This benchmark is a nice idea, but actually you are testing your tool with some hunderts of classes in it, but not F3 itself. FatFree does not combine all your app classes into one single file. It just has 6!! core classes (not 500) in the framework base file and they are needed anyways, so it’s still a good way to save IO operations. All other files are optional plugins. The problem is that you reflect your results directly with F3, which has pretty much nothing in common. Like you said in your benchmark, the performance just get hurt when you have lots of big classes in one file, where you don’t even need all of them. The performance difference in such a small file like F3′s base is just as slightly as not measurable.. and so it’s negligible. I just wanted to clarify this, as i saw other blogs already quoting you and this false essence that F3 could have made something wrong here.

    Reply
    1. admin Post author

      My post wasn’t targeted agaisnt Fat-Free. I said that the difference with XCache is negligale and yes with the aount of classes that Fat Free core has its pretty much the same. I think Fat Free is a nice framework and as you see I kinda wanted to to borrrow one of its ideas.

      Reply
  2. NewUser

    Hej, I’m trying to register on your forum, but seems there’s some problem. I’ve never got the password and throws maintenance errors…I really need to contact with you, because i want to use this framework in production, for a quite big project.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>