PDA

View Full Version : An intriguing coding challenge...



Jaqui
01-15-2008, 11:11 PM
In a recent article I read, the author of the article was asking why common business applications / website scripts do not effectively thread or spread the processing load over multiple cores / cpus.

I figure, here is a good challenge to play with for the year, code in c or c++ a cgi script community website, that uses a database for it's back end. To make sure it uses threading or multiple cpus / cores, the db engine should be included in the code for the script, so connecting to an external engine such as mysql would defeat the purpose.

Things to watch for in execution:
cross site scripting exploits. // hint, absolute urls work better than relative in avoiding these.
sql injection exploits

Goals:
Effective threading / cpu allocation
complete community site, with news, blogs, searching, discussion forums. [ bnet.com or techrepublic.com as examples ]
plugin framework for adding each section as modules
api documentation for module development
complete admin section to add and configure modules as well as domain specific settings.

This is just my own personal project / self challenge for the year. I figure why not put it out so if it catches anyone else's interest they can have a project that isn't a quick bit of coding.

Jaqui
01-15-2008, 11:17 PM
for what it's worth, I'm planning on actually putting the completed app online to test it for full functionality and security.

CornedBee
01-16-2008, 03:38 AM
This looks very misguided to me. Multi-threading any given task gives you a performance overhead. The smaller the tasks that you distribute over multiple CPUs, the higher the overhead.

Server-side applications already make excellent use of multiple CPUs/cores. The granularity level is the request. Every request gets its own thread. Because requests ought to be mostly independent, the synchronization overhead is very low. Multi-threaded web servers like, well, every single one that is in production do a very good job at this stuff. Web application programmers would do a considerably worse job, given that
1) they don't have the excellent request granularity,
2) they don't have years and years of refining a single, relatively static goal (a web server), but instead write a multitude of different applications (the web apps), and
3) on average, web application developers have considerably less experience with multi-threading issues than the guys working on the server.

While this may give you performance gains on systems where there's never enough requests to get all cores busy, the approach doesn't scale. In high-traffic systems, the net performance would be worse. So basically, you get a performance gain on the systems that don't need it.

Sang-drax
01-16-2008, 12:10 PM
I completely agree with Cornedbee, why parallelize web applications when the web server already executes each request in its own thread? There's nothing to gain here, unless there are very few requests and each request takes a very long time to process, which is very untypical for web applications.

abachler
01-16-2008, 03:43 PM
I concur with Cornedbee. Ahmdal's law basically limits the performance you will get out of a web server.

Rashakil Fol
01-16-2008, 06:41 PM
Why anybody would want to make one of these in C or C++ is beyond me.