ARCJSCHMIDT.de
Thoughts from a developer//entrepreneur.
Founder of AETROS.com

open-source

php|Propel1 php|Propel2 php|CDS php|PHP-PM php|php-pm-httpkernel web|Jarves.io web|css-element-queries js|jQuery-selectBox php|PropelBundle php|Propel Sandbox js|angular2-localStorage js|angular-es6-annotations php|php-rest-service php|topsort.php xxx|BetterQuitJobBundle js|angular-typescript-decorators c++|node-core-audio php|optimistic-locking-behavior php|change-logger-behavior

github.com/marcj twitter.com/MarcJSchmidt plus.google.com/+MarcJSchmidt RSS xing.com/profile/MarcJ_Schmidt

PHP High-Performance - Follow Up with Symfony/Jarves.io and PHP-PM

16 April 2016, by Marc

This is a follow up article on “Bring High Performance Into Your PHP App”, which went quiet viral with over 100k visits. This does not only show that many people still struggle with PHP and its performance, but also that people are highly interested in a solution to this kind of issues. PHP-PM could be one solution. But first things first.

Over two years later since my blog post about high-performance things have changed dramatically.

  1. PHP7 came out - and with it a huge performance increasement
  2. PHP-PM got massively more stable and is way faster
  3. First solutions pop up using Symfony in a long running process
  4. PHP-PM is now a thing with over 1,6k stars on github.com/php-pm/php-pm

When I hacked together some lines of code back then in 2013 I never though that this kind of application style would ever succeed in the PHP world. Not only because shared-nothing paradigm is so much powerful, but also because a lot of libraries out there aren’t designed nor intended to run continuously in one php process.

However, things have changed there as well. PHP-PM concentrated as first goal to work with Symfony, because it has a beautiful abstracted HttpKernel Component that makes the whole implementation easier than thought.

So, since we have with HttpKernel a well defined interface that tells us how to query our application and how a response looks, we built a HttpKernel adapter for PHP-PM: github.com/php-pm/php-pm-httpkernel. What it does in a nutshell:

  1. Bootstraps your application (Same as in app.php but without request stuff)
  2. Converts a incoming ReactPHP Request to a Symfony Request
  3. Calls the HttpKernel of the actual application with that request
  4. Converts the Symfony Response to a ReactPHP Response
  5. Sends the response to the client

That sounds pretty easy and straightforward - well, it wasn’t. We had to deal with varieties of issues:

  1. PHP session management sets its own cookie using setcookie, which isn’t included in the Symfony response object and PHP is obviously never detecting a new session again, once started - so we had to fix that as well
  2. We needed to reset a hell of a lot services in Symonfy to prevent memory leaks
  3. We even had to replace the internal native storage service to allow session regeneration

This all feels quiet hacky, but at the end it pays off and works very well. In fact, even a really big application like Jarves.io based on Symfony works pretty well now and can be considered as working alpha.

Although there are still other issues to keep in mind and need to be implemented, it overall already feels like a solution one day becoming stable and a way PHP developer will run PHP.

Open issues

ReactPHP HTTP does not full HTTP/1.1

Most important lack is currently file uploading: But the ReactPHP Team is working on it.. Thanks to the ReactPHP Team and all contributor for your work!

Not all libraries are going to work out of the box

… due to memory leaks. However, PHP-PM implemented auto-respawning of workers that ran out of memory, so this shouldn’t be much of a big deal, but it doesn’t prevent requests from crashing when they hit the max memory. To prevent that beforehand, we implemented also a max-requests per worker option.

This should keep things fairly simple: If you get a lot of memory limit issues then you have several options:

  1. Search for the memory leak and fix it
  2. Reduce the max-requests per worker or give it more memory

Even more issues: github.com/php-pm/php-pm#issues

The Good Things

After the bad things, now the good things: PHP-PM just works. I started using it at not only in the Jarves product itself but also on the jarves.io website. After several days and thousands of requests -> no single issue. So, this is probably one of the first successful production use of PHP-PM.

Another good thing: It is so fast, people think its a static website. But it obviously isn’t: Jarves is a highly dynamic CMS, the website of it builds the documentation and navigation dynamically based on the Jarves code. However, without very clever caching strategies this wouldn’t be possible.

It’s ready

It turns out, that PHP-PM + Jarves makes your application/website so fast, you get even 5ms average response time on a crappy 14$ (2gb ram and 1 slow cpu core) virtual server. That is really a big thing and will definitely cut the costs of bigger websites when they would use PHP-PM. Jarves shows it very clearly: It’s possible to build sophisticated applications in PHP using PHP-PM - This means: PHP is ready for the low-cost high-performance market.

Jarves + PHP-PM = Performance boost

I started building Jarves several years ago without PHP-PM in mind. Once I began to tune the performance, I started looking at the HttpKernel of Symfony and started to figure out how I can improve things. PHP-PM were born. Since that I invested a lot of time making such a sophisticated application like Jarves even possible with PHP-PM. You saw already the results above - which are quiet impressive.

However, this all wasn’t that easy and was much work. I had to think about a lot of things I want to share with you:

Design issues to keep in mind

Service Container: Services

Since Symfony’s Kernel runs continuously in the PHP process, all of your services do as well. This is very important to know, because when you do something in your service that holds some cache or other data directly in your service class without deleting it after the service call, it will be in the RAM until the process dies. What this concretely means:

  1. If not necessarily needed you should delete set class variables after usage.
  2. Do not initialize stuff too eager, because it might be they are never used, but will nonetheless block that memory.
  3. Your service becomes stateful when you don’t reset it, which means the same service is used for every incoming request again and again. Keep that in mind.

Symfony HttpKernel EventListener

This is almost the same like with services. However, in event listeners you have conditions that could happen through the fact, that the whole kernel is handling more than one master-request. This means:

  1. Your listeners will handle many master requests
  2. If you register your security listener before the security listener from Symfony (which checks session stuff) you might run into troubles, because Symfony’s session won’t die after a request is sent. So it might be, that your listener detects a valid session, which isn’t valid. Symfony security listener updates its state only in its listener, so never check security and session stuff before this listener.
  3. Never store the actual Request object, always use RequestStack, because it wouldn’t be updated in the next request.
  4. Also never store the Session object directly. Use TokenStorage instead.

Caching

Caching is a thing most developer don’t consider as too complicated. You only have to write a cache and query it, it couldn’t be much of a big dael, he? Well, it’s not so easy. For simple applications, that do not run in a load balanced scenario (where your application runs on many machines instead of only one), PHP-PM can become an issue.

For example if you have a caching abstraction that uses cache server like memcache or redis and you query bigger amount of data, you probably hold that result in memory for the current request, so the cache does not need to be retrieved every time it is requested. This behavior is going to fail in PHP-PM, because you need to clear that internal cache of the cache after a master-request is sent. I recommend for caching always a invalidation mechanism, so other PHP-PM workers are informed when worker x changed some cache. Rule of thumb: Pretend with PHP-PM that your application is running in a load balanced scenario, where you need to inform other instances about cache changes.

Statefullness

You should never ever build a global state of your application. Since a PHP-PM worker process (where your application lives in) can die every time, you can and should not rely on global variables nor other states.

What is PHP-PM now?

After I told you some issue you should keep in mind when you develop applications with PHP-PM, how could PHP-PM help you? PHP-PM added some features in the last months. Many of those features will help you writing applications in a long running fashion.

  1. Auto-Restart of died workers (due to fatals, memory leaks, uncaught exceptions, out of resources ids, …)
  2. Max-Requests per worker (helps your with memory leaks)
  3. Hot-Code reload (which isn’t a real hot-code reload as PHP does not support it, however, it detects file changes of loaded PHP files and restarts all workers when necessary)
  4. Added way better HttpKernel and Symfony support, so you should be able to just call ppm start in your symfony projects and stuff should just work™.

You should never use PHP-PM as standalone http server, although it’s very practical for local development. Use always a reverse proxy in front of it like NGiNX. NGiNX is responsible for serving static files and PHP-PM for serving dynamic content your actual PHP application.

Performance

Meanwhile, a lot of benchmarks pop up that tested PHP-FPM vs PHP-PM. Most of them are difficult to value.

Some rules of thumbs:

  1. PHP-PM is not going to improve your application if your request handling is much slower than its bootstrap (Symfony bootstrap in particular)
  2. You get only the maxium performance when you use the whole power of a long running process: Cache actual data in PHP arrays and invalidation on distributed caches like memcache/redis.
  3. Massive performance improvement is only given for applications that will a) cache, b) aren’t much slower than the bootstrap itself and c) do heavy lifting not in controllers = utilize the php application process.

This means in particular:

  1. Do heavy lifting in the bootstrap (AppBundle::boot()), not on every incoming request (onKernelRequest or controller call)
  2. Cache all the necessary things in PHP arrays and do not cache useless stuff (see Services Container and Caching section above)
  3. Spot code that could be moved to the bootstrap and move it.

Some other notes in performance

  1. PHP7 improved PHP-PM+Jarves by roughly 30-40%
  2. PHP-PM does have even higher performance on Linux due to unix sockets
  3. Best performance is given for Symfony with: PHP7 + --app-env=prod --debug=0 + a high numbers of workers.
  4. Since Laravel is also based on HttpKernel, it got a massive performance improvement as well.

IMHO this whole approach makes HHVM not necessary anymore, because there is no noticeable performance difference between PHP-PM with PHP7 and PHP-PM with hhvm.

What’s next?

So, the proof is done. It’s possible to use Symfony in long running processes and get with it a massive performance improvement. What’s next? Well we as the community need to make it more stable and use it.

Start contributing

If you have by any chance the willing of contributing to a exciting time, start so please in projects like PHP-PM or ReactPHP. Both are the key to get this whole approach alive. Let us save money in the long term so we don’t need to buy many very expensive servers just to throw out HTML for a website in PHP. :)

If you have any issues using PHP-PM in your application, don’t hesitate to file an issue at github.com/php-pm/php-pm/issues.