Speed Test: Revolution vs 0.9.6

[Guest]

Welcome, Guest. Please login or register.
Did you miss your activation email?

1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[imi]

What I did
I've installed locally (on my own computer) two versions of modx - revolution (alpha-4) and 0.9.6.1
Then i've created empty templates in both versions and made 1 page with the following content:

Code:

[^s^] * SQL:[^qt^]/ [^q^] * parse: [^p^] * total: [^t^]

Then called the page (something like http://localhost/) 10 times by pressing F5

Results:

MODx 0.9.6

Code:

cache * SQL:0.0016 s/ 0 * parse: 0.0430 s * total: 0.0445 s
cache * SQL:0.0016 s/ 0 * parse: 0.0414 s * total: 0.0429 s
cache * SQL:0.0015 s/ 0 * parse: 0.0404 s * total: 0.0419 s
cache * SQL:0.0015 s/ 0 * parse: 0.0381 s * total: 0.0396 s
cache * SQL:0.0015 s/ 0 * parse: 0.0380 s * total: 0.0395 s
cache * SQL:0.0015 s/ 0 * parse: 0.0430 s * total: 0.0445 s
cache * SQL:0.0019 s/ 0 * parse: 0.0457 s * total: 0.0476 s
cache * SQL:0.0015 s/ 0 * parse: 0.0456 s * total: 0.0471 s
cache * SQL:0.0015 s/ 0 * parse: 0.0516 s * total: 0.0532 s
cache * SQL:0.0015 s/ 0 * parse: 0.0463 s * total: 0.0478 s

modx Revolution:

Code:

* cache * SQL:0.0000 s/ 0 * parse: 0.1244 s * total: 0.1244 s
* cache * SQL:0.0000 s/ 0 * parse: 0.1509 s * total: 0.1509 s
* cache * SQL:0.0000 s/ 0 * parse: 0.1557 s * total: 0.1557 s
* cache * SQL:0.0000 s/ 0 * parse: 0.1623 s * total: 0.1623 s
* cache * SQL:0.0000 s/ 0 * parse: 0.1634 s * total: 0.1634 s
* cache * SQL:0.0000 s/ 0 * parse: 0.1572 s * total: 0.1572 s
* cache * SQL:0.0000 s/ 0 * parse: 0.1242 s * total: 0.1242 s
* cache * SQL:0.0000 s/ 0 * parse: 0.1519 s * total: 0.1519 s
* cache * SQL:0.0000 s/ 0 * parse: 0.1621 s * total: 0.1621 s
* cache * SQL:0.0000 s/ 0 * parse: 0.1790 s * total: 0.1790 s

Сonclusion
0.9.6 three times faster then revolution when using empty templates without content! Disappointing result.

The Question
How Revolution will handle complicated templates with many snippets calls? Will it still be usable? 

[atma]

It;s strange. I tried it on remote VPS and get for revolution (page with loremipsum content; template minimal from MODx docs):
cache * SQL:0.0000 s/ 0 * parse: 0.0599 s * total: 0.0599 s
cache * SQL:0.0000 s/ 0 * parse: 0.0595 s * total: 0.0595 s
cache * SQL:0.0000 s/ 0 * parse: 0.0607 s * total: 0.0607 s
...
0.9.6 a little bit slowly...

[rthrash]

Perform the same test but make a site with 500, 2500 and 20,000 pages with more than empty content. Being really fast at doing nothing isn't a good test. What you'll likely find is that for smaller sites, Evolution/096 is marginally faster, though sub-tenth of a second times is pretty quick. For larger sites, Revo just keeps on performing and doesn't degrade as quickly with large sites like happens with the legacy version.

[imi]

Ok, the next step - 500 documents.
I've retrieved 500 html-files from an existing site and imported them into Revo and 0.9.6
Templates used: Revo - empty, 0.9.6 - standart demo template "MODxCSS"

Retrieving 150Kb file...

0.9.6 result:

Code:

MySQL: 0.0675 s, 10 request(s), PHP: 0.6642 s, total: 0.7317 s, document retrieved from database.
MySQL: 0.0702 s, 2 request(s), PHP: 0.2674 s, total: 0.3377 s, document retrieved from cache.
MySQL: 0.0514 s, 2 request(s), PHP: 0.3756 s, total: 0.4270 s, document retrieved from cache.
MySQL: 0.0447 s, 2 request(s), PHP: 0.3313 s, total: 0.3760 s, document retrieved from cache.
MySQL: 0.0364 s, 2 request(s), PHP: 0.3454 s, total: 0.3818 s, document retrieved from cache.

Revolution result:

Code:

* database * SQL:0.0000 s/ 0 * parse: 1.3227 s * total: 1.3227 s
* cache * SQL:0.0000 s/ 0 * parse: 0.8971 s * total: 0.8971 s
* cache * SQL:0.0000 s/ 0 * parse: 0.9577 s * total: 0.9577 s
* cache * SQL:0.0000 s/ 0 * parse: 0.9442 s * total: 0.9442 s
* cache * SQL:0.0000 s/ 0 * parse: 0.9335 s * total: 0.9335 s

Remind that i used "MODxCSS" template for 0.9.6  - it has Wayfinder and Ditto calls... with the blank template it shows total = 0.25 - 0.33 s

P.S. I see that atma posted the different result and it seems strange for me. So please someone else make such test and post result here...

[Sylvaticus]

For my part, I don't see the reasson to do this tests on revolution alpha. I guess (I'm shure) when modx revolution comes out stable it will be more than fast enough. At least for me. 

[dev_cw]

I have to agree with Sylvaticus, I don't really see the point of this test at this time (it is a valid test but I just don't think this is the best time to do it). I am not a big fan of these types of diagnosis I don't think that they really say much. Also as far as comparing the two you are not sure that the measurements are recording the same processes. Maybe in 096 the measurements are less accurate.

[imi]

At least for me

If you are going to use Revo for a homepage, use it. But I do not think that SUCH system is developing for such purpose.

I am not a big fan of these types of diagnosis I don't think that they really say much. Also as far as comparing the two you are not sure that the measurements are recording the same processes. Maybe in 096 the measurements are less accurate

One second parse time says much. REALY MUCH. Of cause the processes are different, but the Revo process is less effective. And i don't understand where i lost the measurement  accuracy... but it may well be so... That's why I ask you all to do this test.

And it's the right time to do this test because  at the moment the developers can (may be) analyse the problem and make corrections in the core of Revo. It will be more difficult to correct the stable release...
I remember that many people complained that 0.9.6 is too slow when having many documents. My test shows that Revo is much slower. Isn't it a problem?

Please understand me - I do not want to abuse Revo or its developers. I want to help them to find a (bottleneck) problem and correct it.

[OpenGeek]

When we get to beta and freeze feature development, we will change focus from flexibility and scalability (in which Revolution will win hands down) to optimization, but not until then.  I've done extensive profiling of this code over the past two years, and am very aware of how it will perform in a variety of deployment scenarios, from simple shared server accounts, to grid server accounts, to complex load-balanced environments with memcached (which 0.9.6 cannot take advantage of without a lot of custom work).  I also have ideas of how to optimize it further without blocking further innovation or limiting scalability/flexibility, so this is definitely going to improve before (and after) beta.

That said, at this point, imi is correct: 0.9.6 on baseline raw speed tests is about 3x faster than Revo alpha.  But as the quantity of both Resources and scripts being executed per page increase, that speed advantage declines exponentially until at a certain point, Revo, without using any extra cacheing features, will begin to outperform the legacy code.  In addition, the use of opcode caches like eAccelerator, should make Revo further outperform the legacy code, which would not be able to take advantage of such systems with the same efficiency.

IMO, some folks may find the current raw speed advantage of the legacy architecture enough to justify choosing it over Revolution, and with small, simple sites, that may indeed be the best solution for now.  But I prefer the robustness and flexibility of developing and maintaining things with the new architecture; I'll be able to take advantage of more internal features and more external facilities for better overall scalability and performance with Revolution.

[shamblett]

I agree with OpenGeek here, the key is scalability, if you design with this in mind your on to a winner, OK it may not work so well at the lower end but these are the trade offs you have to consider. Apps like MODx by their nature are constrained by the environment in which they run i.e they depend on webservers like apache and databases like MYSQL. If you are on say a 16 or 32 processor machine with 16 to 32GB of RAM(which is not that uncommon) it wont' help you much if apache/mysql only use 1 processor to do everything, I know they don't but you'd be surprised by the number of apps that still do this, tweaking MODx won't help much here.

Ideally MODx should be able to run on everything from mobile phones to grid installations and adapt itself to the environment it finds itself in. I know this is a bit holy grail'ish but this doesnt mean these things shouldnt be considered and designed in. You can rarely have everything.

PS to OpenGeek, interesting use of the word 'legacy' in your post, I don't think the 0.9.6 series is legacy just yet, I think it may be around for some time!
 

[imi]

OpenGeek, thank you for reply.
I have no more questions

[woundedbeer]

Interesting, but not a very good real world test.

I've migrated one site to Revolution but didn't fully finish it as I am waiting for the 1.0 release. When that comes out, I'll finish the migration to 1.0 and run some tests to compare.

I have about 4500 documents on 0.9.6 and it is doing pretty well because I hacked the document parser and moved snippets to files to keep the siteCache file smaller. You do notice the speed decrease in 0.9.6 as you have more docs, but you can do a lot to increase performance.

I've recently configured Apache with gzip, moved my server to Amazon EC2 on a medium instance (~1.7GB Ram and 5 compute units), and installed eaccelerator. Huge boost! Still have some MySQL optimization to do and might move MySQL to a dedicated instance on EC2. When Revolution comes out, I'll get memcached going and it will smoke!

But as mentioned, I think the difference will be seen when the stable release is out and you compare larger/more complex sites.