Catalyst vs Rails vs Djan
realzhang
|
1#
realzhang 发表于 2006-11-05 17:45
Catalyst vs Rails vs Djan
Catalyst vs Rails vs Djan
请看：http://letsgetdugg.com/feed/view/Catalyst_vs_Rails_vs_Django_Cook_off Today I began working on a new project and decided to benchmark Catalyst and Rails for fun. See how my new favorable framework does against Rails. I was a bit shocked at the results though. I guess this is worth mentioning in hope Catalyst can improve in it's Accessor Generation code. So here are the results: Benchmark System Celeron 1.8Ghz 1 Gig of Ram FreeBSD-6 Interpreters: Ruby - 1.8.5 Perl - 5.8.8 Frameworks: Catalyst - 5.7003 Rails - 1.1.6 Run as: Lighttpd: 1.4.13 FCGI: 3 max proc Benchmarked as: ab -n 1000 -c 100 http://siteurl.com/ h3. Some background I specifically turned off sessions and did not use ActiveRecord/DBIC to keep it as fair as possible between the two frameworks. Both frameworks were run under Lighttpd and FCGI. I tried to keep this as apples to apples as possible. So lets take a look at the results! Rails: Server Software: lighttpd/1.4.13 Server Hostname: wansanity.com Server Port: 9090 Document Path: /main/index Document Length: 2142 bytes Concurrency Level: 100 Time taken for tests: 18.261 seconds Complete requests: 1000 Failed requests: 0 Broken pipe errors: 0 Total transferred: 2296892 bytes HTML transferred: 2143288 bytes Requests per second: 54.76 [#/sec] (mean) Time per request: 1826.10 [ms] (mean) Time per request: 18.26 [ms] (mean, across all concurrent requests) Transfer rate: 125.78 [Kbytes/sec] received Connnection Times (ms) min mean[+/-sd] median max Connect: 74 885 1742.9 138 11785 Processing: 172 661 1216.8 173 8195 Waiting: 84 661 1216.8 173 8194 Total: 172 1547 2123.8 330 11893 Percentage of the requests served within a certain time (ms) 50% 330 66% 1354 75% 2786 80% 3106 90% 4297 95% 6279 98% 8216 99% 9285 100% 11893 (last request) Thats 54 connections / sec which is great. I have seen it peak at 70 connections/sec which is just awesome! Catalyst: Server Software: lighttpd/1.4.13 Server Hostname: wansanity.com Server Port: 80 Document Path: / Document Length: 2232 bytes Concurrency Level: 100 Time taken for tests: 43.503 seconds Complete requests: 1000 Failed requests: 0 Broken pipe errors: 0 Total transferred: 2401300 bytes HTML transferred: 2238490 bytes Requests per second: 22.99 [#/sec] (mean) Time per request: 4350.30 [ms] (mean) Time per request: 43.50 [ms] (mean, across all concurrent requests) Transfer rate: 55.20 [Kbytes/sec] received Connnection Times (ms) min mean[+/-sd] median max Connect: 75 322 808.5 93 6028 Processing: 269 3804 851.8 3928 6754 Waiting: 192 3804 851.7 3928 6754 Total: 269 4126 1178.5 4186 10293 Percentage of the requests served within a certain time (ms) 50% 4186 66% 4384 75% 4404 80% 4424 90% 5025 95% 6422 98% 7194 99% 7709 100% 10293 (last request) 22 connections / sec not exactly what I expected from a framework built on top of the fast Perl Interpreter. Being a bit disappointed with the results, I investigated further. So here are the perl dprof results. %Time ExclSec CumulS #Calls sec/call Csec/c Name 0.00 0.605 4.128 1512 0.0004 0.0027 NEXT::AUTOLOAD 0.00 0.373 0.373 25794 0.0000 0.0000 Class::Accessor::Fast::__ANON__ 0.00 0.235 0.235 1177 0.0002 0.0002 NEXT::ELSEWHERE::ancestors 0.00 0.211 0.225 1 0.2107 0.2253 YAML::Type::code::BEGIN 0.00 0.184 5.182 86 0.0021 0.0603 Catalyst::Engine::HTTP::_handler 0.00 0.177 0.205 2583 0.0001 0.0001 File::Spec::Unix::canonpath 0.00 0.164 0.309 1942 0.0001 0.0002 File::Spec::Unix::catdir 0.00 0.156 2.408 3201 0.0000 0.0008 Catalyst::Action::__ANON__ 0.00 0.134 0.739 73 0.0018 0.0101 base::import 0.00 0.129 0.136 5904 0.0000 0.0000 Class::Data::Inheritable::__ANON__ 0.00 0.109 0.814 7 0.0155 0.1163 main::BEGIN 0.00 0.108 0.108 1323 0.0001 0.0001 HTTP::Headers::_header 0.00 0.101 0.116 10 0.0101 0.0116 Template::Parser::BEGIN 0.00 0.101 0.334 11 0.0092 0.0304 Catalyst::Engine::BEGIN 0.00 0.101 0.295 1264 0.0001 0.0002 Path::Class::Dir::stringify It seems like the main bottleneck in Catalyst 5.7003 is Next. Jrockway was kind enough to post some new code into Catalyst's trunk for me to try; a new replacement for Next - C3. Here are the results with the C3 Plugin from Trunk %Time ExclSec CumulS #Calls sec/call Csec/c Name 0.00 0.211 0.233 1 0.2106 0.2330 YAML::Type::code::BEGIN 0.00 0.135 0.135 8035 0.0000 0.0000 Class::Accessor::Fast::__ANON__ 0.00 0.126 0.721 73 0.0017 0.0099 base::import 0.00 0.109 0.116 10 0.0109 0.0116 Template::Parser::BEGIN 0.00 0.108 0.805 7 0.0155 0.1150 main::BEGIN 0.00 0.093 0.106 7 0.0133 0.0152 Catalyst::Engine::HTTP::Restarter: :Watcher::BEGIN 0.00 0.090 0.105 1023 0.0001 0.0001 File::Spec::Unix::canonpath 0.00 0.085 0.326 11 0.0077 0.0296 Catalyst::Engine::BEGIN 0.00 0.081 0.905 196 0.0004 0.0046 Catalyst::execute 0.00 0.069 0.120 8 0.0087 0.0150 Catalyst::Plugin::Server::XMLRPC:: Request::BEGIN 0.00 0.064 1.639 444 0.0001 0.0037 next::method 0.00 0.061 0.313 32 0.0019 0.0098 Catalyst::BEGIN 0.00 0.054 0.216 7 0.0077 0.0309 Template::Config::load 0.00 0.054 0.189 4 0.0135 0.0473 HTTP::Body::OctetStream::BEGIN 0.00 0.054 0.388 4 0.0135 0.0970 Gambit::BEGIN So there you have it, the results with the C3 Plugin. It only made a slight difference by pushing the Catalyst benchmark score to 25 connections / sec. I hope this benchmark can get some changes put into place for Catalyst's next release. Conclusion It seems like Rails is roughly 62% faster than Catalyst at this time. Keep in mind this benchmark does not take into account the ORM performance. This benchmark tests how quick the frameworks themselves dispatch methods and render views. Also take into consideration when choosing a framework you need to look at the problem at hand. Catalyst can feed off Perl's vast CPAN resource library. Catalyst has features that Rails does not have. Catalyst's DBIC ORM supports multi-column primary keys and can do relationship mapping just by reading the schema! You don't even have to bother writing any has_many belongs_to definitions! I am going to have to take a look into Django see how well it fairs in this benchmark. Perhaps an update on this? Update Django Results Server Software: lighttpd/1.4.13 Server Hostname: fab40 Server Port: 9090 Document Path: / Document Length: 2235 bytes Concurrency Level: 100 Time taken for tests: 13.643 seconds Complete requests: 1000 Failed requests: 0 Broken pipe errors: 0 Total transferred: 2409769 bytes HTML transferred: 2253459 bytes Requests per second: 73.30 [#/sec] (mean) Time per request: 1364.30 [ms] (mean) Time per request: 13.64 [ms] (mean, across all concurrent requests) Transfer rate: 176.63 [Kbytes/sec] received Connnection Times (ms) min mean[+/-sd] median max Connect: 76 483 1068.1 101 8666 Processing: 190 744 726.3 571 6088 Waiting: 93 744 726.4 572 6088 Total: 190 1227 1414.2 692 9606 Percentage of the requests served within a certain time (ms) 50% 692 66% 972 75% 1209 80% 1445 90% 3282 95% 4020 98% 6414 99% 8113 100% 9606 (last request) 72 connections / sec! Amazing and the winner! And anyone that disagrees with this can go ahead and look at the code for all three projects. I have the least experience with django for your information mst Please don't kill me' Many thanks go out to jrockway to helping me point out the root cause of the bottleneck in Catalyst. |