Thursday, August 18, 2011

PECL-Memcache standard loadbalancer for Nginx

In a previous post i have already introduced my Nginx loadbalancer, which is compatible with the PECL-Memcache consistent hashing. Unfortunately we discovered that the implementation of the consistent hashing on the PHP side is quite inefficient since it rebuilds the hashring on every request, and in the case of our PHP framework even for each Memcache server on each request. So we had to decide between a Memcache proxy like Moxi to implement the consisten hashring, or just forget about the advantages of a consistent hash and save the additional hop of a moxi. The conclusion was to just use the standard PECL-Memcache loadbalancer, since this is probably the most resource efficient way to implement a cache, and then just live with having to regenerate some caches in case we add/remove Memcache instances.
So in order to make the Nginx find cache values which have been stored by PHP into Memcache, i wrote another Nginx loadbalancer which is simply copying the behaviour of the PECL-Memcache standard loadbalancer.
Example:
 70 upstream profilememcache {
 71     hash_key $memcached_key;
 72     server 10.20.0.27:11211;
 73     server 10.20.0.26:11211;
 74 }

-----

 570         location /profile {
 570         set $memcached_key $uri;
 571         error_page      500 404 405 = /fallback_php;
 572         memcached_pass  profilememcache;
 573         }
In the above example you see that the memcached_pass can be used just as always and in the upstream definition you simply need to add the parameter "hash_key" together with based on what the loadbalancer should do the hashing on.
Unfortunately its really important that the order of the Memcache servers in the upstream definition is the same as the order in which the Memcache servers are being added to the Memcache object in the PHP code. If the positions of the servers are not the same the whole thing won't work.

1 comment:

  1. Hello Mauro,

    Iam very interested in this module. We are using a PHP-Application Cluster and for caching a Memcache-Cluster. To store the data in the memcache we use:

    http://www.php.net/manual/de/book.memcache.php

    I have some questions about that:

    1. To use your nginx module (NginxHttpMemcacheStandardBalancer) I have to set the php directive "memcache.hash_strategy" to standard? Right?

    2. If I have multiple php-servers and multiple memcache-servers -> do all php-servers will store the same key on the same memcache server?

    3. What happends if the PECL-Memcache standard loadbalancer changes the behaviour where to store the keys? Or is that unlikely?

    Thank you very much for you help.

    Best regrads,
    Tobias from Berlin, Germany

    ReplyDelete