I found some time to push the current versions of DB and ORM modules to github.

You can find the 3.0 DB module branch on https://github.com/dracony/PHPixie-DB/tree/3.0 and the ORM module is available on https://github.com/dracony/PHPixie-ORM/tree/3.0.

As I said before the database module is pretty much done, apart from me still having to add in documentation comments. Another thing is support for the DISTINCT modifier which I need for ORM.

The ORM module is being rewritten entirely too. It will retain most of what you already got used to, but here are some new features that will be added (some are more or less already implemented):

• Multiple strategies for different situations. For example if you are using tables in the same database the relationships will be processed using IN subqueries and joins, but if your model tables reside in different databases PHPixie will first extract item ids from one model and use them to query the other.
• Ability to specify relationships between SQL tables and MongoDB collections. This is permitted because of the multiple approaches described previously.
• You’ll be allowed to specify relationship conditions in a simple fashion. E.g. you can directly query deep relationship paths:php?start_inline=1 $query ->has('fairy.tree.location', 'forest') ->or_has('fairy.flower.name', 'Daisy') ->or_has('fairy.tree.name', 'Oak') - Relationship query optimization. The above query if using the multiple query approach would in other frameworks load ids for the fairy.tree.location condition and fairy.tree.name separately and would require you to rewrite this query in a different fashion (e.g. using callbacks) to prevent such behaviour. PHPixies ORM has a fully implemented optimizer to take care of this itself. Of course you can always groups conditions manually if you want to spare optimizer time.

And now something totally new, relationships won’t be defined inside classes anymore, they get a special configuration file for that. Also you won’t have to specify belong_to and has_many relationships separately. Instead you define familar oneToMany and OneToOne relationships like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
//inside the config file
return array(
   array(
       'type' => 'oneToMany',
       'owner' => array(
             'model' => 'author'
        ),

       'items' => array(
             'model' => 'post'
        ),
   ),
   array(
       'type' => 'manyToMany',
       'left' => array(
             'model' => 'post'
        ),

       'right' => array(
             'model' => 'tag'
        ),
   )
);

//Obviosly these relationships allow a lot more options to configure,
//this is just a minimal setup.
 

Now isn’t that much more readable and simple?

There is (and has been for a long time) some discussion about Laravels’ approach to hiding the composition of core components behind a facade. Now, for those who don’t know yet, in OOP world a facade is an object that simplifies working with a complex system buy providing a simple interface. Before talking about this let’s take a better look about how they are implemented inside that framework:

Laravel takes the concept in a bit of a different direction and instead of providing a single facade for the whole system it creates multiple mini-facades for every part of the system. Their official docs say that Facades provide a “static” interface to classes that are available in the application’s IoC container. I would say that they need to update it to say that this is how their facades work, not how they are generally thought of. There has been a lot of discussion on whether their static approach (and I don’t know why they put brackets around the word “static” since it is indeed static) is beneficial or harmful, so I won’t talk about that either, there is a bigger question to discuss here.

Should developers dumb down their frameworks?

Every developer wants his product to be used, and does as much as possible to broaden his target audience. One of the best things he can do is providing a nice learning curve that would let people gradually get more and more acquainted with his product. Unfortunately this may sometimes lead to rather bad chices that might hurt in the long run. The success of rather complex frameworks like Symfony2 should have taught us that keeping things complex but correct isn’t necessary a hurtful decisions. I’ve known a lot of people switching back an fourth between “easy” frameworks like Laravel and Yii, but I’ve seen only a couple leave Symfony2 after they’ve got to grips with it. Even now a lot of causes for choosing Laravel on reddit go along the lines of “Symfony was too complex”. Now myself I don’t think Symfony is perfect, but I think it makes a good example of the point I’m trying to get across.

Dumbing the framework down and hiding it behind a facade is in my opion a bad idea, since when the developer reaches the level of understnding required to get IoC an dependency injection he will find himself refactoring a huge pile of working code. The framework shouldn’t ease things down, on the contrary it should enforce best practices, and even though it will make the user hate it for the first week, he will fall in love with it after a month.

Good documentation better than simplification

So how should a framework ensure a nice learning curve while requiring high cde standarts? The answer lies in providing a good documentation. People dump harder frameworks because they say they can get same results in less time using a different one. This is not a code issue, but a documentation problem that should be solved by providing extensive tutorials and practical examples.

What if you have great docs but some users are too lazy to read those? Well, you can’t win them all, can you.

Today I wrote the final test for the 3.0 version of the Database module. I started this thinking it would take me only a couple of weeks, but it spanned for about 2 months due to the humongous amount of refactoring and rewriting from scratch. What we have now is a scalable and extendable database library which can be used even outside of PHPixie itself (so if you’re forced to use some other framework or CMS you’ll be able to bring your favourite database library with you).

I’ll release it into the master branch when I’m done with ORM and will alter existing module to work with updated libraries. But if you’d like a sneak peek you can take a look at the 3.0 branch on github (be warned that I haven’t added documentation for it yet, since I’m expecting some slight modifications to it while working on the ORM module). So now let’s take a look at what’s new.

From an average user standpoint:
I have already posted about the advanced support for MongoDB conditional queries here, but if you’re too lazy to click that link, here is a short explanation: MongoDB will not use indexes if you use complex logical queries like “(A or B) OR ( C and (D or E))”, so PHPixie will expand such queries to forms that will use indexes.

Different approaches to query building, to fit your needs:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$query
    ->where('name', 'Trixie')
    ->or_where('name', 'Tinkerbell')
    ->where_not('id', '>', 7)
    ->having('count','<', 5)
    ->or_having('count', 7);
;

//Or a simpler way:
$query
    ->where('name', 'Trixie')
    ->_or('name', 'Tinkerbell')
    ->_and_not('id', '>', 7)
    ->having('count', '<', 5)
    ->_or('count', 7);
//Notice how the _or shorthand rememebrs the current context
//(e.g. it's used both for 'where' and 'having')
 

Different approaches to nested logic:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
$query
    ->where('name','Trixie')
    ->or(function($builder){
        $builder
            ->_and('id',7)
            ->_or('id',5)
    });
//WHERE name = 'Trixie' OR ( id = 7 OR id = 5 )

//Or if you don't like callbacks (also usefull for dynamic query building, e.g. grids)
$query->where('name','Trixie');
$query
    ->start_where_group('or')
        ->_and('id',7)
        ->_or('id',5)
    ->end_where_group();
 

Special operators for comparing columns with other columns:

1
$query->where('fairies.id','*=','pixies.id');
 

Ability to add raw expressions while still using prepared statements:

1
2
3
$expr = $this->db->expr('concat(name, ?)', array('test'));
$query->where($expr, 'Trixietest');
//WHERE concat(name, ?) = 'Trixietest';
 

Finding rows with NULL, without knowing it’s NULL beforehand:

1
2
3
$category_id = null;
$query->where('category_id', $category_id);
//WHERE category_id IS NULL
 

Complex conditions support for JOIN … ON queries using same syntax as where and having

1
2
3
4
5
6
7
$query->join('pixies');

//Using column comparison by default
$query->on('fairies.id','pixies.id');

//But we can also add some other conditions
$query->on('fairies.count','*>','pixies.count');
 

From the advanced developer standpoint:

• Amazingly light, 57 KB of code
• Query builders don’t contain any parsing logic by themselves, isntead each driver uses its own Parser service, thus making queries lightweight and memory efficient. Using parsers as services makes debugging and extending them much easier. Also since each parser is instantiated only once it makes them even more efficient.
• Parsers are decoupled into Condition Group parsers (like WHERE statements) and individual operator parsers. Operator parsers are easily extendible by adding methods to them, so you can add a bunch of your own frequently used snippets there.
• The dependency wireing is done in separate driver classes, so at no point (apart from raising exceptions) do any of the classes instantiate objects on their own using the new statement. This makes it so much easier to replace system classes with your own.
• MongoDB queries are parsed into special Runner instances, that are easy to inspect if an error occurs
• SQL databases that use PDO get their own Adapters and Parser instances, making adding support for other PDO databases trivial

Everything has been unit tested, resulting in 166 tests and 1151 assertions

I hope this will get you as excited for the new release as I am. Hopefully the updated ORM moduel is not far away =)

Today I finished the hardest part of making a MongoDB driver, support for nested logic like AND, OR XOR. By default MongoDB will not use indexes if the quesry has nested $or elements and will perform a full table scan. The only way to use indexes is to modify the query so that it is a single $or query with nested $and queries, like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//This query:
{
    $and:[
        {$or:[A,B]},
        {$or:[C,D]}
    ]
}

//Has to be
{
    $or:
        [
            {$and:[A,C]},
            {$and:[A,D]},
            {$and:[B,C]},
            {$and:[B,D]}
        ]
}
 

Not only that, the worst part is that you can’t negate a subquery, making it even harder to implement things like XOR. Now I wanted a full query builder support in PHPixie (with any kind of negations and nested conditions), so that the developer would not have to think about inflating his queries to get all possible combinations. I’ve just finished writing a code that does just that. It will even try to optimize the end result, by removing duplicated queries and operands. The alorythm I used is efficient enough for making inflecting even of long and complex queries really really fast.

It took me a HUGE amount of time to come up with efficient ways of doing this, especially the negation part (hello De Morgan’s laws), but that part is done. So you all can expect a super useful MongoDB query builder when the ne DB module is released.

Thinking about ways of further improving PHPixie I started looking at other projects for inspiration. For example the Fat-Free framework boasts on being contained in a single file. This got me thinking about making a tool for merging all project classes together with vendor libraries into a single for performance boost. MAking such a tool is a fairly trivial task, but still I wanted to be sure it would actually be usefull, so I decided to benchmark autoloading classes with composer vs combining them into a single file.

The setup was fairly simple:
1) Generate a set number of classes
2) Build a composer classmap for them to speed up autoloading
3) Combine them into a merged file
4) Benchmark the amount of time spent instantiating the classes
5) Repeat steps 1-4 for a number of classes ranging from 1 to 500

To generate classes that would simulate realistic IO each class would get a random number of methods with random number of lines in them. Finally the class generator would look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
function build_method($class,$i) {
  $str = " function {$class}_{$i}() {\r\n";

        for ($i = 0;$i<rand(15,30);$i++){
                $str.= ' $i = '.rand(100, 200).' + '.rand(10, 90).";\r\n";
        }

        $str.= "}\r\n";
        return $str;
}

function build_header() {
        $str="<?php\r\n";
        $str.= "namespace A;\r\n";
        return $str;
}
function build_class($i){

        $extends = $i%10 > 0?"extends A".($i-1)." ":"";

        $str= "class A$i $extends{\r\n";
        for($j = 0; $j < rand(4, 9); $j++)
                $str.= build_method("A$i", $j);
        $str.= "}\r\n";
        return $str;
}

$merged = "";
for($i=0;$i<$argv[1];$i++){
        $class = build_class($i);
        $merged.=$class;
        file_put_contents( __DIR__."/A/A{$i}.php", build_header().$class);
}

file_put_contents( __DIR__."/merged.php", build_header().$merged);
 

To add even more realism I made it so that some classes are children of other classes, this way instead of instantiating every class to load it, I can just instantiate classes A9 and A19 to load all classes from A0 to A19 in 2 calls. From here it was just a matter of setting up 2 php, files, one that would execute testcases using autoloaded clasess, the other using the merged ones.

So here are the results:

Benchmarking loading of all generated classes for each test

As wee can see on the first chart, with XCache disabled the Combined approach outperforms Autoloading as the number of classes increases, which is quite logical, the performance difference is about 0.03ms with 500 classes which is really negligible. Now let’s try turning XCache on:

Loading classes with opcache enabled

Xcache pretty much levels everything by caching the opcodes, thus negating any reason to combine classes into a single file. I’ll go even further and prove that combining classes into a single file will actually hurt performance. The reason being is that in both tests we loaded all of the project classes, this doesn’t happen in real life though.

If you have a large project that has lots of different logic in it I’d say that on each request only about 10% of all project libraries are needed. In this case autoloading will work much faster, since it doesn’t load useless code and hog memory like the combined approach would.

So here is my final benchmark. For this one I load only 10% of generated classes (so for 500 generated classes in a test only 50 are being used):

Loading only 10 percent of generated classes

As we can see Autoloading dramatically outperforms the merged file approach. Now I don’t mean to say that Fat-Free isn’t a fast framework, I just think that this particular aspect of it isn’t really usefull. So I guess PHPixie will not be getting a “combine your classes into a single file” tool after all.

So is there anything that we still can do to improve autoloading performance? Yes! We can use composer properly.

1
2
3
4
5
6
$loader = require 'vendor/autoload.php';

//This is the WRONG approach
//because adding your files like this
//you won't benefit from the classmap cache
$loader->add('Acme\\Test\\', __DIR__ );
 

The correct approach is to put this in your composer.json:

1
2
3
4
5
6
7
8
9
{
    "autoload": {
        "psr-0": {"": "classes/"}
    },
}

//Now if you run "php composer.phar update -o"
//A classmap cache will include your project files too
//thus (slightly) boosting autoload speed
 

Validation module has just been massively updated. I’ve made a move from the array based definition to a somewhat more verbose OOP style. You will also find some tasty new features like step-by-step validation, throwing exceptions and redone conditional rules. You can read the details in the documentation, but here I will show you a quick demo of what it is capable of (and yes I believe this is the most flexible validation system I’ve ever seen)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
//Initialize validator
$validate = $pixie->validate->get($some_data);

/* Basic stuff */
$validate->field('username')

    //Adding multiple rules
    ->rules('filled','alpha_numeric')

    //Adding a rule that requires extra parameters
    ->rule('min_length',8)

    //Setting a custom error message
    ->error('Username is invalid');

//Adding password and password confirmation
$validate->field('password')
    ->rule('filled');
$validate->field('password_confirm')
    ->rule('same_as', 'password');



/* Conditional rules */



//The sate field is required...
$validate->field('state')
    ->rule('filled')
    ->rule('in', $us_states)

    //...only if the 'country' field is 'US'
    ->condition('country')
        ->rule('filled')
        ->rule('equals', 'US');



//'city' is required if the country is other than US
$validate->field('city')
    ->rule('filled')
    ->condition('country')
        ->rule('filled')
        //Notice how we can inverse the rule with '!'
        ->rule('!equals', 'US');



/**
 * Some rules should throw exceptions, instead of attacker-friendly error messages
 */



$validate->field('secret_token')
    ->rule('filled')

    //Yes, it supports callbacks
    ->rule('callback',function($token){
        //validate token, return true if valid
    })
    ->throw_on_error("Someone is trying to hack us");



/**
 * Step by step validation
 * E.g. validate some rules only if previous ones are valid
 */



$validate->field('username')
    ->rule('filled','alpha_numeric');


//The second 'true' parameter means those rules will be applied if the 
//previous ones were valid.
//In our case there is no need to check invalid usernames for uniqueness
$validate->field('username',true)
    ->rule('callback',function(){
        //Check if username is unique
    });



/* Combine with use() statement for even more greatness */
$parent = null;
$validate->field('parent_id')
        ->rule('callback', function($parent_id) use ($parent){
            $parent = ... //Get the parent model
            return $parent->loaded();
        })
        ->error("Parent doesn't exist");


//Since we passed parent in a use() statement in previous callback
//It will be already present in the second one.
//Furthemore the following rule will run only if the first one was valid
$validate->field('parent_id', true)
        ->rule('callback', function($parent_id) use ($parent){
            //Check if $parent can be a parent for the current item
        })
        ->error('The parent specified is invalid');



/* Finally get results */

//check if valid
$validate->valid();

//get errors
$validate->errors();
 

PHPixie skeleton project can now be installed using a single command (well almost).
All you need is to get composer:

1
php -r "eval('?>'.file_get_contents('http://getcomposer.org/installer'));"
 

And then run the following line to create a project for you:

1
php composer.phar create-project phpixie/project . 2.*-dev --prefer-dist
 

This will create a phpixie project in your current directory.

You should also update the autoloader to optimize class loading:

1
php composer.phar update -o
 

Running the above command will also optimize autoloading of your custom classes too, so it might be usefull to run it once more before deploying your website.

Using PHP templating has a significant drawback of being unable to automatically escape your variables when they are being displayed. So if you had avariable like “Fairy

1
2
3
4
5
6
7
8
9
// inside a view
<!-- Old way -->
<div><?php echo $fairy->name; ?></div>

<!-- Using a helper. Rather long-->
<div><?php $helper->output($fairy->name);></div>

<!-- Using a shortcut -->
<div><?php $_($fairy->name); ?></div>
 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
// /classes/App/View/Helper
namespace App\View;

class Helper extends \PHPixie\View\Helper{

    protected $aliases = array(
    '_' => 'output',
    '_r' => 'repeat'
    );
    public function repeat($str, $times) {
        for($i = 0;$i<$times; $i++)
            echo $str;
    }
}
 

1
2
<!-- Will print <br/> 3 times -->
<?php $_r('<br/>',3) ?>