CI4 performance regression #6889
Comments
joanhey
added
the
bug
kenjis
removed
the
bug
|
How do we run this CI4 app? |
|
Clone https://github.com/TechEmpower/FrameworkBenchmarks in your computer. In the main directory: Verify tests: Bench in local: Debug mode, for watch in your browser that is running OK: |
|
You can visualize your local bench results in: or https://tfb-status.techempower.com/share The results are generated in the dir You also can run other fw bench to compare: cakephp, laravel, ... |
|
The results from CI3 The actual running with CI4 : When finish, I'll send the logs. |
|
Here you can see the plaintext test numbers for CI3: Remember than in local will be lower. |
|
Some days before I updated Slim, with good results. You can check it here: |
|
CI3's code does not work. $ ./tfb --mode verify --test codeigniter
...
techempower/mysql:latest: ---> Running in f296bd6da0f6
techempower/mysql:latest: W: GPG error: http://repo.mysql.com/apt/ubuntu bionic InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 467B942D3A79BD29
techempower/mysql:latest: E: The repository 'http://repo.mysql.com/apt/ubuntu bionic InRelease' is not signed.
techempower/mysql:latest: Removing intermediate container f296bd6da0f6
techempower/mysql:latest: Docker build failed; terminating
techempower/mysql:latest: Traceback (most recent call last):
techempower/mysql:latest: File "/FrameworkBenchmarks/toolset/utils/docker_helper.py", line 55, in __build
techempower/mysql:latest: raise Exception(token['errorDetail']['message'])
techempower/mysql:latest: Exception: The command '/bin/sh -c apt-get update > /dev/null' returned a non-zero code: 100
techempower/mysql:latest: Build time: 2m 39s
A fatal error has occurred
Traceback (most recent call last):
File "/FrameworkBenchmarks/toolset/run-tests.py", line 244, in main
any_failed = benchmarker.run()
File "/FrameworkBenchmarks/toolset/benchmark/benchmarker.py", line 58, in run
self.docker_helper.build_databases()
File "/FrameworkBenchmarks/toolset/utils/docker_helper.py", line 315, in build_databases
tag="techempower/%s" % db)
File "/FrameworkBenchmarks/toolset/utils/docker_helper.py", line 55, in __build
raise Exception(token['errorDetail']['message'])
Exception: The command '/bin/sh -c apt-get update > /dev/null' returned a non-zero code: 100
Shutting down (may take a moment) |
|
Failed the In some minutes I'll try in my local. PD: Do you remember me ?? https://github.com/kenjis/php-framework-benchmark/commits?author=joanhey |
|
I copied CI3 code into the latest repository and ran the benchmarks on my macOS. https://www.techempower.com/benchmarks/#section=test&shareid=4e82ab70-1a3a-4e64-a9cf-010c67062d6a&test=json |
|
I found the database schema. |
|
The regression is global, so you can test it with the json and plaintext tests. Also, after checking the numbers and code from CI3 and CI4, I check an architectural problem. |
|
Perhaps using the ORM will be fixed. |
|
Perhaps the regression is only in the database connection: But for the above line included always, we see the regression in all tests. And it isn't recommended to use it so, as will be very bad for use with any persistent platform using workers and childs. Like Workerman, Swoole, ... |
|
I can make some changes, and send a new PR to the bench. |
|
Added a fast change, later you can change it. But we don't have a reference from CI3, because always was loading the bd also. |
|
Still with the last change is slow: Before: Only gain in json and plaintext 100-200 req/s CI3 json test: |
|
Try |
|
All the frameworks using preload gain ~1-2%. |
|
Try this change: |
|
Hello, We recently moved to CI 4.2 in one of our project. Before that, we were using CI 4.1. As soon as we updated, we started seeing an increase in CPU usage on our servers. We have hundreds of routes, we might be able to optimize some things on our side, but I thought pitching in what we found could help in finding this regression. We ran on 4.2 for two days. During that time, we were trying to figure out what was causing the spike. I marked that period in red in the screenshot below.
We ran a few tests locally and used xDebug's profiler to see if there was any bottleneck in our application. We noticed an increase in execution time in RouteCollection.php. We reverted the following changes from #6644. More specifically: https://github.com/codeigniter4/CodeIgniter4/pull/6644/files#diff-b28efb4cd802e8a3ead515befe9f46254b6cc9d17ab1beeb8a42a16cff69d283L1247-R1258 As soon as we did, the CPU usage of our servers went back to normal (purple line in the screenshot). Like I said, it might not be what you're looking for, but hopefully that helps. |
|
@patches Thank you for your investigation. For reference, how many routes are defined? |
|
@kenjis We're talking about 1571 routes. Which is quite a lot. |
|
I have confirmed that the time to register a lot of routes has increased significantly since v4.2.8. |
|
See also my post on the forum: https://forum.codeigniter.com/showthread.php?tid=88083 The performance degradation is partly solved I notice. But still not as it used to be I think. You can also try to run the benchmarks of this person: https://github.com/myaaghubi/PHP-Frameworks-Bench. Maybe try running it on CI3 vs CI4 on the same PHP version, without OPCache and maybe also with OPCache+JIT? |
|
CI4 performance has not improved at all. I believe this is due to the fact that CI4 has considerably more functionality than CI3. Also, there does not seem to be a particular need for performance tuning and no one is doing it. In fact, CI4 is probably fast enough. The only way to improve performance is to benchmark (profile) and find bottlenecks, and eliminate them. |
|
The Hello World benchmark only measures the minimal overhead of the framework. Other processes are much larger in the actual application. So even if the Hello World benchmark is faster, like Symfony, it is not the same as if the actual app is faster. In any case, benchmarks are meaningless unless you measure them in your own environment. The numbers will change depending on various conditions. |
|
That said, I would like CI4 to be faster and would welcome performance tuning work and PRs for it. |
|
A A framework can have more functionality, but for that is the autoload. PD: Also it is using a lot of memory !!! |
|
I got the Hello World benchmark with Apache on Ubuntu 22.02. $ wrk -t10 -d5s http://localhost/ci3.1.13/
Running 5s test @ http://localhost/ci3.1.13/
10 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.13ms 1.25ms 20.01ms 96.12%
Req/Sec 1.04k 229.18 1.54k 57.40%
52703 requests in 5.10s, 8.10MB read
Requests/sec: 10334.21
Transfer/sec: 1.59MB$ wrk -t10 -d5s http://localhost/ci4.3.7/
Running 5s test @ http://localhost/ci4.3.7/
10 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 6.81ms 3.24ms 46.62ms 82.29%
Req/Sec 151.34 35.28 340.00 62.30%
7609 requests in 5.10s, 1.50MB read
Requests/sec: 1492.09
Transfer/sec: 301.88KB$ php -v
PHP 8.1.2-1ubuntu2.13 (cli) (built: Jun 28 2023 14:01:49) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.1.2, Copyright (c) Zend Technologies
with Zend OPcache v8.1.2-1ubuntu2.13, Copyright (c), by Zend Technologies$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.3 LTS" |
|
By doing the following, I got about 4240 rps. It is 2.4 times slower than CI3.
$ wrk -t10 -d5s http://localhost/ci4.3.7/
Running 5s test @ http://localhost/ci4.3.7/
10 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 2.50ms 1.59ms 23.41ms 91.58%
Req/Sec 428.97 91.32 710.00 63.24%
21625 requests in 5.10s, 4.27MB read
Requests/sec: 4240.65
Transfer/sec: 858.00KB |
|
The Config performance can be improved by #7696. |
|
See #8017 (comment) |
|
Would you mind to share if we need to change any specific configuration options for these kind of caching? I now try to go to all the file changed in these PRs, however this doesn't help me much/isn't efficient. Bottom-line: Is there something I need to be aware of, as an user? Or it is only internal stuff :).? |
|
See the "Configuration" in the comment. |
|
Profiling
diff --git a/app/Config/Services.php b/app/Config/Services.php
index df7c8ad086..8289cbf614 100644
--- a/app/Config/Services.php
+++ b/app/Config/Services.php
@@ -2,6 +2,8 @@
namespace Config;
+use CodeIgniter\Autoloader\FileLocator;
+use CodeIgniter\Autoloader\FileLocatorCached;
use CodeIgniter\Config\BaseService;
/**
@@ -17,16 +19,18 @@ use CodeIgniter\Config\BaseService;
* method format you should use for your service methods. For more examples,
* see the core Services file at system/Config/Services.php.
*/
-class Services extends BaseService
+class Services extends \CodeIgniter\Config\Services
{
- /*
- * public static function example($getShared = true)
- * {
- * if ($getShared) {
- * return static::getSharedInstance('example');
- * }
- *
- * return new \CodeIgniter\Example();
- * }
- */
+ public static function locator(bool $getShared = true)
+ {
+ if ($getShared) {
+ if (! isset(static::$instances['locator'])) {
+ static::$instances['locator'] = new FileLocatorCached(new FileLocator(static::autoloader()));
+ }
+
+ return static::$mocks['locator'] ?? static::$instances['locator'];
+ }
+
+ return new FileLocator(static::autoloader());
+ }
}
diff --git a/public/index.php b/public/index.php
index 8cf5ce347c..e5557cd706 100644
--- a/public/index.php
+++ b/public/index.php
@@ -49,8 +49,8 @@ if (! defined('ENVIRONMENT')) {
}
// Load Config Cache
-// $factoriesCache = new \CodeIgniter\Cache\FactoriesCache();
-// $factoriesCache->load('config');
+$factoriesCache = new \CodeIgniter\Cache\FactoriesCache();
+$factoriesCache->load('config');
// ^^^ Uncomment these lines if you want to use Config Caching.
/*
@@ -79,7 +79,7 @@ $app->setContext($context);
$app->run();
// Save Config Cache
-// $factoriesCache->save('config');
+$factoriesCache->save('config');
// ^^^ Uncomment this line if you want to use Config Caching.
// Exits the application, setting the exit code for CLI-based applications |
|
Profiling plain |
|
With branch Plain $ wrk -t10 -d5s http://localhost/CodeIgniter4/
Running 5s test @ http://localhost/CodeIgniter4/
10 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 7.00ms 3.26ms 34.53ms 82.66%
Req/Sec 147.05 35.81 290.00 61.39%
7410 requests in 5.10s, 123.37MB read
Requests/sec: 1452.83
Transfer/sec: 24.19MBPlain $ wrk -t10 -d5s http://localhost/CodeIgniter4/
Running 5s test @ http://localhost/CodeIgniter4/
10 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 4.91ms 2.32ms 26.29ms 81.95%
Req/Sec 209.90 49.85 530.00 51.50%
10482 requests in 5.10s, 174.80MB read
Requests/sec: 2055.98
Transfer/sec: 34.29MB |
If you use the caching features, the hello world benchmark will be probably improved by 2 times. Also, it is just showing "hello world" or something static. |
|
Property access is 6 times faster than <?php
namespace App\Controllers;
use Config\Services;
class Home extends BaseController
{
public function index(): string
{
$iterator = new \CodeIgniter\Debug\Iterator();
$that = $this;
$iterator->add('$this->request', static function () use ($that) {
$path = $that->request->getPath();
});
$iterator->add('Services::request()', static function () {
$path = Services::request()->getPath();
});
$iterator->add('service(\'request\')', static function () {
$path = service('request')->getPath();
});
return $iterator->run(300000);
}
} |
This doesn't account for the actual class instantiation, though, does it? Once you're accessing |
|
No. That is code in a controller. Services already instantiated the Request object. |
|
FYI, if we change the parent class of --- a/app/Config/Services.php
+++ b/app/Config/Services.php
@@ -17,7 +17,7 @@ use CodeIgniter\Config\BaseService;
* method format you should use for your service methods. For more examples,
* see the core Services file at system/Config/Services.php.
*/
-class Services extends BaseService
+class Services extends \CodeIgniter\Config\Services
{
/*
* public static function example($getShared = true) |
|
Measured including the time to inject into the constructor, using <?php
namespace App\Controllers;
use Config\Services;
class Home extends BaseController
{
public function index(): string
{
$iterator = new \CodeIgniter\Debug\Iterator();
$typography = Services::typography();
$iterator->add('constructor injection', static function () use ($typography) {
$obj1 = new DiClass($typography, new \stdClass());
$obj1->foo();
});
$iterator->add('Services', static function () use ($typography) {
$obj2 = new SlClass(new \stdClass());
$obj2->foo();
});
return $iterator->run(300000);
}
}
class DiClass
{
public function __construct(private $typography, private $config)
{
}
public function foo()
{
$this->typography;
}
}
class SlClass
{
public function __construct(private $config)
{
}
public function foo()
{
Services::typography();
}
} |
|
Retrieving objects from <?php
namespace App\Controllers;
use InvalidArgumentException;
class Home extends BaseController
{
public function index(): string
{
$iterator = new \CodeIgniter\Debug\Iterator();
$config = config('App');
$container = new Container();
$container->set('App', $config);
$iterator->add("config()", static function () {
$config = config('App');
});
$iterator->add('Container', static function () use ($container) {
$config = $container->get('App');
});
return $iterator->run(300000);
}
}
class Container
{
private array $instances;
public function set(string $name, mixed $val): void
{
$this->instances[$name] = $val;
}
public function get(string $name): mixed
{
if (! isset($this->instances[$name])) {
throw new InvalidArgumentException('No such entry.');
}
return $this->instances[$name];
}
}If we use a global array variable. diff --git a/app/Controllers/Home.php b/app/Controllers/Home.php
index 5934333309..cda6c0375a 100644
--- a/app/Controllers/Home.php
+++ b/app/Controllers/Home.php
@@ -6,6 +6,21 @@ class Home extends BaseController
{
public function index(): string
{
- return view('welcome_message');
+ $iterator = new \CodeIgniter\Debug\Iterator();
+
+ $config = config('App');
+ global $container;
+ $container['App'] = $config;
+
+ $iterator->add("config()", static function () {
+ $config = config('App');
+ });
+
+ $iterator->add('Container', static function () {
+ global $container;
+ $config = $container['App'];
+ });
+
+ return $iterator->run(300000);
}
}
diff --git a/public/index.php b/public/index.php
index 8cf5ce347c..f3166d38be 100644
--- a/public/index.php
+++ b/public/index.php
@@ -68,6 +68,9 @@ $app->initialize();
$context = is_cli() ? 'php-cli' : 'web';
$app->setContext($context);
+// Global container.
+$container = [];
+
/*
*---------------------------------------------------------------
* LAUNCH THE APPLICATION |
I won't want to bother you too much, it could be me, but I'm unable find the FileLocator caching (where the idea is to change I did however found the Config factory section in the user-guide: https://codeigniter4.github.io/userguide/concepts/factories.html#id21. Which also explain how to enable that feature: https://codeigniter4.github.io/userguide/concepts/factories.html#how-to-enable-config-caching. But again maybe I'm just stupid, but I couldn't find the FileLocator cache stuff. The only mention of |
|
We have two git branches for development. I meant it was the in-progress user guide on the In |
|
ow I see. Looking forward for the official 4.5 release 🥳 ! |
|
Small feedback on: https://github.com/codeigniter4/CodeIgniter4/blob/4.5/user_guide_src/source/installation/deployment.rst#adding-htaccess This is talking about server {
///....
root /var/www/somedomain.com/html/public;
location / {
try_files $uri $uri/ /index.php$is_args$args;
}
location ~ \.php(?:$|/) {
include snippets/fastcgi-php.conf;
}
location = /robots.txt {
allow all;
log_not_found off;
access_log off;
}
location = /favicon.ico {
log_not_found off;
access_log off;
}
location ~ /\. {
deny all;
return 404;
access_log off;
}
}Ps. my # regex to split $uri to $fastcgi_script_name and $fastcgi_path
fastcgi_split_path_info ^(.+?\.php)(/.*)$;
set $path_info $fastcgi_path_info;
# Check that the PHP script exists before passing it
try_files $fastcgi_script_name =404;
include fastcgi.conf;
## fastcgi settings
# Set PHP handler
fastcgi_pass unix:/run/php/php8.2-fpm.sock;
fastcgi_index index.php;
# Increase timeout
fastcgi_read_timeout 600;
fastcgi_intercept_errors on;
# Disable buffering (for uploading files)
fastcgi_request_buffering off;
# Increase memory size of the buffer segments (used for the payload of the response)
# 32 x 16k
fastcgi_buffers 32 16K;
# Increase memory buffer for HTTP response header
fastcgi_buffer_size 32k;
# fastcgi params
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $path_info;
fastcgi_param HTTPS on;
fastcgi_param modHeadersAvailable true; # Avoid sending the security headers twice
fastcgi_param front_controller_active true; # Enable pretty But you can argue that only accepting |
|
@melroy89 Thank you for the feedback. But it is off topic. This issue is on CI4 performance. |
Sorry about that. I know. I just noticed it, because it was on the same "Deployment" page. Thanks I will look into that other page and create PRs if needed. |
PHP Version
8.1
CodeIgniter4 Version
4.2.10
CodeIgniter4 Installation Method
Composer (as dependency to an existing project)
Which operating systems have you tested for this bug?
Linux
Which server did you use?
fpm-fcgi
Database
MySQL 8
What happened?
After update to CI4 in the TechEmpower bench.
CI3 PHP 8.1 vs CI4 PHP8.1 (with JIT)
Req/s
Please, could anybody review the code in the bench.
If after review the code, still get this performance, exist another problem.
Thank you
Steps to Reproduce
https://tfb-status.techempower.com/
I can help, only ask me.
Expected Output
Similar or higher numbers than CI3.
Not 5 times slower.
Anything else?
PR that update to CI4: TechEmpower/FrameworkBenchmarks#7686
Final code to review: https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/PHP/codeigniter
Previous code with CI3: https://github.com/TechEmpower/FrameworkBenchmarks/tree/e062401d0dbc660ed2e7c969ddbcc84e66ae5824/frameworks/PHP/codeigniter
When finish the run, I'll send the logs.
The text was updated successfully, but these errors were encountered: