Thread: Clusters and scalability

  1. #16
    It's full of stars adrianxw's Avatar
    Join Date
    Aug 2001
    Posts
    4,829
    Not for the first time, I haven't the faintest idea what you lot are talking about.
    Wave upon wave of demented avengers march cheerfully out of obscurity unto the dream.

  2. #17
    Mayor of Awesometown Govtcheez's Avatar
    Join Date
    Aug 2001
    Location
    MI
    Posts
    8,823
    >> Not for the first time, I haven't the faintest idea what you lot are talking about.

    Children's show from the early 90s. I think it's actually still on, in some form.

    Alright, I got the skinny from my brother. According to him, their magic belt buckles or something lame like that turns them into Power Rangers, and when they're in the Power Ranger state, they can call their Zords just by asking for them, and usually jumping into the air. He also said the Green/White ranger was the only one that called by playing his flute of gayness.

  3. #18
    Registered User
    Join Date
    Jan 2002
    Location
    Vancouver
    Posts
    2,212
    Power rangers is the worst TV show I have ever seen.

  4. #19
    Registered User
    Join Date
    Jul 2002
    Posts
    913
    You can use something like OpenMosix.

    It load into the (Linux/Unix) kernel and lets you split programs into tasks. The best way to do it is to have a program with a lot of threads, theres also some special libraries you can use(normally c or fortran, could be anything).

    Each computer is a node and you set up groups. Every group gets a switch and every node gets that many nics.

    This is fun stuff to play with, a good reason to keep all those all pcs or get a cheap Lindows pc(like $200 bucks, you can build a nice node for alittle over $300).

    The normally dont have monitors, video cards, disk drives(some times just a hard drive), keyboards or mice. You can have them boot off the network card(load teh os over the network, no hard drive/cdrom) if you set up a tftp server, i havent played with that(my school is starting soon, i will soon).

    Its kool stuff to look into. But theres bunch of different kind of cuper computer. This is the "poor mans supercomputer".



    lol, power ranges....

  5. #20
    Registered User
    Join Date
    Sep 2002
    Posts
    11
    Download a copy of OSCAR. It is a collection of tools used to set up a Beowulf Cluster. I tried it at work a few years ago. We get better performance from our software using 10 PC (Pentiums 400 - 700 MHz, with two NIC per, ie we ran two seperate LANS, one for data passing, the other for control) than from the SGI machine that software was targetted for. The PC were old office Gateway machine, resale value of about $1000 total. The SGI was an 8 CPU Power Onyz costing $800,000.

    To get the best performance, you will need to optimize your code to take advantage of multiple node. For your message passing, MPI is becoming the standard and should work on most versions of Unix/Linux. PVM is still used, but I have never tried it.

    BTW: openMosix will work with RH9.0. The last time I had looked at the website, it had only been tested with RH7.3.
    Ed Bowen

  6. #21
    Banned nickname_changed's Avatar
    Join Date
    Feb 2003
    Location
    Australia
    Posts
    986
    Thanks, I was looking at an article on intel and they said beowulf clusters are very powerful, but code must be specially written to take advantage of it. openMosix on the other hand doesn't require recompiling the code, so if I ever do try out clustering, I think thats the way I'll do it.

    Wow, I can't believe 10 PC's outperformed the SGI, it must be pretty powerful stuff. Thanks for all the info guys!

  7. #22
    Registered User
    Join Date
    Sep 2002
    Posts
    11
    Remember that SGI specializes graphics processing and it is hard to find something better. The software I was working on at that time did not take advantage of the SGI's strengths.
    Ed Bowen

  8. #23
    Banned nickname_changed's Avatar
    Join Date
    Feb 2003
    Location
    Australia
    Posts
    986
    Ohh ok, but still, anything worth $800,000 should be able to out pace $1000 worth of old computers. I've been reading the openMosix documentation and as soon as I install linux again (in a few months) I'll hook up my other computers and use their power together.

  9. #24
    Banned nickname_changed's Avatar
    Join Date
    Feb 2003
    Location
    Australia
    Posts
    986
    Okay, onto another question. Lets say I have a web server, with about 100 gig of data. I want to set up a cluster of, say, 6 servers with 1.2ghz athlons.

    Where would I keep the 100 gig of data that needs to be sent out across the net? On all of the servers, or just one? Or, on a seperate data storage server? But wouldn't that mean I'd have a bottle neck at the storage server? Or, would I have everything "mirrored" somehow on all the servers?

    How do sites like microsoft keep all the data on their servers the same? If I had three storage servers with 100gig each, how could I make sure they all held the same files? Would it be some sort of CVS for normal files? This is interesting....

  10. #25
    Me -=SoKrA=-'s Avatar
    Join Date
    Oct 2002
    Location
    Europe
    Posts
    448
    On the OpenMosix documentation, it says that if you're running a webserver or a database server, you can't take advantage of clustering (at least with OpenMosix).
    I have no idea how MS keeps al their data. They probably use a super-secret server that can access data on any computer in a record time and they don't tell anyone obout
    Seriously though, I've no idea.
    SoKrA-BTS "Judge not the program I made, but the one I've yet to code"
    I say what I say, I mean what I mean.
    IDE: emacs + make + gcc and proud of it.

  11. #26
    Registered User
    Join Date
    Sep 2002
    Posts
    11
    For a website, the bottle neck is not in your machine, it is in the ethernet connection.
    Ed Bowen

  12. #27
    Registered User
    Join Date
    Jul 2002
    Posts
    913
    A cluster can be used for a datase.

    The space should probally be in one machine shared withe the rest. Maybe a LVM is possible over ethernet, I have no idea.

    The internet connection would be the problam, the slashdot effect.

  13. #28
    C++ Developer XSquared's Avatar
    Join Date
    Jun 2002
    Location
    Ontario, Canada
    Posts
    2,718
    >>For a website, the bottle neck is not in your machine, it is in the ethernet connection.
    Not if you're using server-side scripting.
    Naturally I didn't feel inspired enough to read all the links for you, since I already slaved away for long hours under a blistering sun pressing the search button after typing four whole words! - Quzah

    You. Fetch me my copy of the Wall Street Journal. You two, fight to the death - Stewie

  14. #29
    Registered User
    Join Date
    Jul 2002
    Posts
    913
    Not if you're using server-side scripting.
    Like PHP or a Database?

    Yeah, a cluster would be nice there.

    Now im wondering about a Etherner-LVM

  15. #30
    Banned nickname_changed's Avatar
    Join Date
    Feb 2003
    Location
    Australia
    Posts
    986
    Yeah thats what I was thinking, a site of mostly PHP and MySQL databases. The PHP on all computers would be easy, but keeping the data in the database the same on all the computers would be hard. Actually, how could you keep all the files for the site on the same computer? Would it be possible to use CVS? If you can't I'm going to invent a way of keeping them all mirrored

Popular pages Recent additions subscribe to a feed