PHP High-Performance - Follow Up with Symfony/Jarves.io and PHP-PM
This is a follow up article on “Bring High Performance Into Your PHP App”, which went quiet viral with over 100k visits. This does not only show that many people still struggle with PHP and its performance, but also that people are highly interested in a solution to this kind of issues. PHP-PM could be one solution. But first things first.
Over two years later since my blog post about high-performance things have changed dramatically.
- PHP7 came out - and with it a huge performance increasement
- PHP-PM got massively more stable and is way faster
- First solutions pop up using Symfony in a long running process
- PHP-PM is now a thing with over 1,6k stars on github.com/php-pm/php-pm
When I hacked together some lines of code back then in 2013 I never though that this kind of application style would ever succeed in the PHP world. Not only because shared-nothing paradigm is so much powerful, but also because a lot of libraries out there aren’t designed nor intended to run continuously in one php process.
However, things have changed there as well. PHP-PM concentrated as first goal to work with Symfony, because it has a beautiful abstracted HttpKernel Component that makes the whole implementation easier than thought.
So, since we have with HttpKernel a well defined interface that tells us how to query our application and how a response looks, we built a HttpKernel adapter for PHP-PM: github.com/php-pm/php-pm-httpkernel. What it does in a nutshell:
- Bootstraps your application (Same as in app.php but without request stuff)
- Converts a incoming ReactPHP Request to a Symfony Request
- Calls the HttpKernel of the actual application with that request
- Converts the Symfony Response to a ReactPHP Response
- Sends the response to the client
That sounds pretty easy and straightforward - well, it wasn’t. We had to deal with varieties of issues:
- PHP session management sets its own cookie using setcookie, which isn’t included in the Symfony response object and PHP is obviously never detecting a new session again, once started - so we had to fix that as well
- We needed to reset a hell of a lot services in Symonfy to prevent memory leaks
- We even had to replace the internal native storage service to allow session regeneration
This all feels quiet hacky, but at the end it pays off and works very well. In fact, even a really big application like Jarves.io based on Symfony works pretty well now and can be considered as working alpha.
Although there are still other issues to keep in mind and need to be implemented, it overall already feels like a solution one day becoming stable and a way PHP developer will run PHP.
ReactPHP HTTP does not full HTTP/1.1
Not all libraries are going to work out of the box
… due to memory leaks. However, PHP-PM implemented auto-respawning of workers that ran out of memory, so this shouldn’t be much of a big deal, but it doesn’t prevent requests from crashing when they hit the max memory. To prevent that beforehand, we implemented also a max-requests per worker option.
This should keep things fairly simple: If you get a lot of memory limit issues then you have several options:
- Search for the memory leak and fix it
- Reduce the max-requests per worker or give it more memory
Even more issues: github.com/php-pm/php-pm#issues
The Good Things
After the bad things, now the good things: PHP-PM just works. I started using it at not only in the Jarves product itself but also on the jarves.io website. After several days and thousands of requests -> no single issue. So, this is probably one of the first successful production use of PHP-PM.
Another good thing: It is so fast, people think its a static website. But it obviously isn’t: Jarves is a highly dynamic CMS, the website of it builds the documentation and navigation dynamically based on the Jarves code. However, without very clever caching strategies this wouldn’t be possible.
It turns out, that PHP-PM + Jarves makes your application/website so fast, you get even 5ms average response time on a crappy 14$ (2gb ram and 1 slow cpu core) virtual server. That is really a big thing and will definitely cut the costs of bigger websites when they would use PHP-PM. Jarves shows it very clearly: It’s possible to build sophisticated applications in PHP using PHP-PM - This means: PHP is ready for the low-cost high-performance market.
Jarves + PHP-PM = Performance boost
I started building Jarves several years ago without PHP-PM in mind. Once I began to tune the performance, I started looking at the HttpKernel of Symfony and started to figure out how I can improve things. PHP-PM were born. Since that I invested a lot of time making such a sophisticated application like Jarves even possible with PHP-PM. You saw already the results above - which are quiet impressive.
However, this all wasn’t that easy and was much work. I had to think about a lot of things I want to share with you:
Design issues to keep in mind
Service Container: Services
Since Symfony’s Kernel runs continuously in the PHP process, all of your services do as well. This is very important to know, because when you do something in your service that holds some cache or other data directly in your service class without deleting it after the service call, it will be in the RAM until the process dies. What this concretely means:
- If not necessarily needed you should delete set class variables after usage.
- Do not initialize stuff too eager, because it might be they are never used, but will nonetheless block that memory.
- Your service becomes stateful when you don’t reset it, which means the same service is used for every incoming request again and again. Keep that in mind.
Symfony HttpKernel EventListener
This is almost the same like with services. However, in event listeners you have conditions that could happen through the fact, that the whole kernel is handling more than one master-request. This means:
- Your listeners will handle many master requests
- If you register your security listener before the security listener from Symfony (which checks session stuff) you might run into troubles, because Symfony’s session won’t die after a request is sent. So it might be, that your listener detects a valid session, which isn’t valid. Symfony security listener updates its state only in its listener, so never check security and session stuff before this listener.
- Never store the actual Request object, always use RequestStack, because it wouldn’t be updated in the next request.
- Also never store the Session object directly. Use TokenStorage instead.
Caching is a thing most developer don’t consider as too complicated. You only have to write a cache and query it, it couldn’t be much of a big dael, he? Well, it’s not so easy. For simple applications, that do not run in a load balanced scenario (where your application runs on many machines instead of only one), PHP-PM can become an issue.
For example if you have a caching abstraction that uses cache server like memcache or redis and you query bigger amount of data, you probably hold that result in memory for the current request, so the cache does not need to be retrieved every time it is requested. This behavior is going to fail in PHP-PM, because you need to clear that internal cache of the cache after a master-request is sent. I recommend for caching always a invalidation mechanism, so other PHP-PM workers are informed when worker x changed some cache. Rule of thumb: Pretend with PHP-PM that your application is running in a load balanced scenario, where you need to inform other instances about cache changes.
You should never ever build a global state of your application. Since a PHP-PM worker process (where your application lives in) can die every time, you can and should not rely on global variables nor other states.
What is PHP-PM now?
After I told you some issue you should keep in mind when you develop applications with PHP-PM, how could PHP-PM help you? PHP-PM added some features in the last months. Many of those features will help you writing applications in a long running fashion.
- Auto-Restart of died workers (due to fatals, memory leaks, uncaught exceptions, out of resources ids, …)
- Max-Requests per worker (helps your with memory leaks)
- Hot-Code reload (which isn’t a real hot-code reload as PHP does not support it, however, it detects file changes of loaded PHP files and restarts all workers when necessary)
- Added way better HttpKernel and Symfony support, so you should be able to just call
ppm startin your symfony projects and stuff should just work™.
You should never use PHP-PM as standalone http server, although it’s very practical for local development. Use always a reverse proxy in front of it like NGiNX. NGiNX is responsible for serving static files and PHP-PM for serving dynamic content your actual PHP application.
Meanwhile, a lot of benchmarks pop up that tested PHP-FPM vs PHP-PM. Most of them are difficult to value.
Some rules of thumbs:
- PHP-PM is not going to improve your application if your request handling is much slower than its bootstrap (Symfony bootstrap in particular)
- You get only the maxium performance when you use the whole power of a long running process: Cache actual data in PHP arrays and invalidation on distributed caches like memcache/redis.
- Massive performance improvement is only given for applications that will a) cache, b) aren’t much slower than the bootstrap itself and c) do heavy lifting not in controllers = utilize the php application process.
This means in particular:
- Do heavy lifting in the bootstrap (
AppBundle::boot()), not on every incoming request (onKernelRequest or controller call)
- Cache all the necessary things in PHP arrays and do not cache useless stuff (see Services Container and Caching section above)
- Spot code that could be moved to the bootstrap and move it.
Some other notes in performance
- PHP7 improved PHP-PM+Jarves by roughly 30-40%
- PHP-PM does have even higher performance on Linux due to unix sockets
- Best performance is given for Symfony with: PHP7 +
--app-env=prod --debug=0+ a high numbers of workers.
- Since Laravel is also based on HttpKernel, it got a massive performance improvement as well.
IMHO this whole approach makes HHVM not necessary anymore, because there is no noticeable performance difference between PHP-PM with PHP7 and PHP-PM with hhvm.
So, the proof is done. It’s possible to use Symfony in long running processes and get with it a massive performance improvement. What’s next? Well we as the community need to make it more stable and use it.
If you have by any chance the willing of contributing to a exciting time, start so please in projects like PHP-PM or ReactPHP. Both are the key to get this whole approach alive. Let us save money in the long term so we don’t need to buy many very expensive servers just to throw out HTML for a website in PHP. :)
If you have any issues using PHP-PM in your application, don’t hesitate to file an issue at github.com/php-pm/php-pm/issues.