Anyone here know about Node.js?

So, I thought about maybe rewriting a site I'm working on using Node.js and all the associated frameworks.

But I have some real questions. It seems like Express will do what I need it to do. It's also great because the code I write will in essence make whitelisting/blacklisting page requests very simple. If a user requests, /users/data, I have to manually code that in otherwise, the request gets sent to a default page. It's all fair enough.

But one of the things is, the pages I request will have dynamically generated content. Now, I understand that Node is single-threaded. To me, this makes it seem ideal to serve static pages.

I'm using Apache + PHP now and one thing I like a lot more is that Apache creates a new thread per page request and then I can have PHP running its own thread populate the HTML and send it back.

I'm scared that if I use Node.js, that main thread will quickly become over-saturated with user instructions.

As I understand it, all Node's really good at is instruction layering with the main thread. You have a single-thread but somehow through the divine magic of animal sacrifice, you have the ability to layer instructions in a much more efficient way so you can receive more page requests while you're fetching the pages of a previous request. Seems very well-designed but generating the HTML I need dynamically would be blocking which makes Apache + PHP seem more ideal.

Thoughts?

Quote:

---

Now, I understand that Node is single-threaded.

---

O_o

Your understanding of "Node.js" is wrong due to being so very incomplete.

Your misunderstanding is actually the standard for "Node.js" newcomers.

I don't want to waste too much time explaining the grit. I'll just simplify.

The code "Node.js" code you would normally write is processed within a single thread. (You can use various modules which let you have fibers, threads, and forking.) The code you would normally write is asynchronous by default. (You can use various modules which are synchronous.) You will not find magic in the "Node.js" code; you'll find either asynchronous primitives from the host environment or synchronous primitives and threads.

The best way to explain is with code. I don't actually use "Express" so the code is just a "Node.js" fragment.

Code:

---

g = [];

fs.readFile('example1.txt', function DoExample1(fError, fContents) {
    g.push('1')
})

fs.readFile('example2.txt', function DoExample2(fError, fContents) {
    g.push('2')
})

---

The files "example1.txt" and "example2.txt" are read by "Node.js" modules with asynchronous interfaces from the host environment or synchronous interfaces from the host environment wrapped by threads. The files "example1.txt" and "example2.txt" may be read any order.

The functions `DoExample1` and `DoExample2` are executed by "Node.js" as if they are the only code being executed. The functions `DoExample1` and `DoExample2` may be executed in any order.

The global array `g` can not be accessed or mutated by the `DoExample1` and `DoExample2` functions at the same time because "Node.js" is will only execute one handler at a time. The file "example1.txt" or "example2.txt" is potentially still being read while the `DoExample1` or `DoExample2` function is being executed.

You should remember the "Node.js" adage "everything except your code runs in parallel".

Quote:

---

Thoughts?

---

The "Node.js" environment and similar do not magic away the cost of complex tasks.

You should be asking "Where does the cost live?" if you are trying to evaluate the "Node.js" environment.

Are your "instructions" costly in terms of waiting on other components? The database, file, and socket primitives provided by the "Node.js" environment are handled asynchronously with the code you write. You aren't going to be waiting on all connections simply because a database query in your code hasn't finished; your code isn't going to execute until the query is complete.

Are your "instructions" costly in terms of compute? You should probably consider other alternatives. You are going to be waiting on all connections if the handler for a finished database connection implements an expensive process.

*shrug*

You can find modules for threading in the "Node.js" environment, but I'd suggest you stick with what you are already learning.

Soma

Hmm... That's interesting.

Just a simple proof of concept right now, I want to ditch Apache and PHP (just for the sake of learning).

Serving static files isn't an issue as of now but one thing I'm having trouble with is directory requests. In Apache, you can use HyperText Access files with the RewriteEngine to send all directory requests to a PHP file which I then use to generate the HTML dynamically. I'm using the PHP to basically make a <table /> with all the directory contents.

For example, here's my .htaccess file in current site's directory root :

Code:

---

RewriteEngine On
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^ generate_directory.php [L]

---

I actually got this code from someone awesome on the PHP board.

How I do this in Node.js is currently beyond me but I'm trying to learn. I guess my misunderstanding of PHP let me believe that if I had like 1000 simultaneous requests for a directory, each request would become serialized as generating the HTML isn't largely I/O bound. Sure, some of it is but not all. All I really need is one call to get the directory contents but after that, it's up to the Node engine to take those contents and make HTML out of it.

I'm just so confused lol.

Edit : It seems like Express can be configured to handle all GET requests so every time a user wants to GET a URL and the URL is a valid directory path, I guess I can just generate the HTML from that. I think I'm doing that thing where I massively overly-complicate things in my head before I even try to do what it is I'm thinking of doing.

Quote:

---

Originally Posted by MutantJohn

I'm scared that if I use Node.js, that main thread will quickly become over-saturated with user instructions.

As I understand it, all Node's really good at is instruction layering with the main thread. You have a single-thread but somehow through the divine magic of animal sacrifice, you have the ability to layer instructions in a much more efficient way so you can receive more page requests while you're fetching the pages of a previous request. Seems very well-designed but generating the HTML I need dynamically would be blocking which makes Apache + PHP seem more ideal.

Thoughts?

---

I have actually thought of recommending NodeJS to you before, but I wasn't sure how demanding the generation of your content would be. Like Soma says, everything in NodeJS happens (or appears to happen) asynchronously, but the code you run, like generating tables or doing math etc will be done in synchronous fashion. JavaScript itself is quite slow (I've run into this problem after implementing a SAT collision mechanism in my JS game engine), but because all the IO is asynchronous, it probably beats the alternatives given you aren't calculating PI for several minutes in your request handler lol.

Quote:

---

Originally Posted by MutantJohn

Edit : It seems like Express can be configured to handle all GET requests so every time a user wants to GET a URL and the URL is a valid directory path, I guess I can just generate the HTML from that. I think I'm doing that thing where I massively overly-complicate things in my head before I even try to do what it is I'm thinking of doing.

---

It sounds like you are talking about something I've heard called a "software router". I don't know if it will help, but below is some code from the book "Eloquent JavaScript", that implements a very basic software router (an unaltered version), with comments by myself. Basically the purpose is allow the user to associate a pattern to search requests for, with a callback function to be invoked when a match is found:

Code:

---

module.exports = (function () {
 
    "use strict";
 
    var Router = function () {
        this.routes = [];
    };

    // Add a 'route' to the software router, where a 'route' should contain parts:
    // 1. method  : the HTTP method that we want to handle.
    // 2. pattern : A regular expression that will be used to test incomming requests.
    // 3. handler : A callback function to be executed if any incomming requests match the pattern
    Router.prototype.add = function (method, pattern, handler) {
        this.routes.push({method: method, pattern: pattern, handler: handler});
    };
 
    Router.prototype.resolve = function (request, response) {

        // extract the path from the request url using 'url' module's 'parse' method
        var path = require("url").parse(request.url).pathname;

        // Return true if any routes in the array can resolve the path
        return this.routes.some(function (route) {

            // Use the routes regular expression to get matching parts in the requested path
            var match = route.pattern.exec(path),
                urlParts;

            // Return false if no matches were found, or if the request type differs from the route type
            if (!match || route.method !== request.method) {
                return false;
            }

            // The first match in the array created by 'exec' is the full path,
            // The rest of the matches are transformed by decoding the URL encoding
            // and stored in a new array.
            urlParts = match.slice(1).map(decodeURIComponent);

            // The callback function is envoked with request, response, and all the pathway matches as arguments
            route.handler.apply(null, [request, response].concat(urlParts));
            return true;
        });
    };
 
    return Router;
 
}());

---

This is meant to be run as part of a request handler, for instance after adding some routes, you would test them in your request handler like this:

Code:

---


function requestHandler (request, response) {
    // If our router doesn't resolve the request, pass it to the file server.
    if (!router.resolve(request, response)) {
        serveFile(request, response);
    }
}

---

It might be unnecessary to write it into your request handler though, if you instead 'subclass' the request handler (save a copy of the the current request handler and call it when resolving url fails). I dislike doing this on my own, because you need to be aware of what's happening or you might overwrite your modules handler by attaching a new one.

Some modules will do this though, for instance if you look around line 113 in the socket.io's 'manager' module:

Code:

---

// reset listeners
this.oldListeners = server.listeners('request');
server.removeAllListeners('request');

server.on('request', function (req, res) {
    self.handleRequest(req, res);
});

---

Back on topic though, keep in mind that the 'Router' module from Eloquent JavaScript is very basic, and does no error checking whatsoever, but I think it illustrates the point. An example of how you might register a pathway handler from the same book:

Code:

---

// Register a handler to be invoke on "GET" requests that have a pathway that looks like "/talks/*"
router.add("GET", /^\/talks\/([^\/]+)$/, function(request, response, title) {
  if (title in talks)
    respondJSON(response, 200, talks[title]);
  else
    respond(response, 404, "No talk '" + title + "' found");
});

---

Hope anything of that was helpful, good luck! :D

Most of that kind of went over my head lol. It's okay, I actually found something kind of relevant : https://www.npmjs.com/package/serve-index

The default CSS is also pretty sexy too. If anything, I was going to try and reading this again : Express routing

Edit :

Just to vent a little bit, how is this better than Apache + PHP again? :P

To me, JavaScript makes a lot of sense on the front-end. Creating interactive web pages is boss. But Node.js doesn't seem all that useful for serving content through a HTTP. Maybe I'm just a noob and I'm encountering a learning curve but darn it all, Apache's pretty sexy and PHP interfaces with it pretty darn well!

Quote:

---

Just to vent a little bit, how is this better than Apache + PHP again?

---

O_o

Are you seriously still swallowing buzzword b......t?

If you don't think a project is a fit for your work, the opinions of other people are irrelevant.

Why would you even consider, again, evaluating a project based on marketing you didn't even try to investigate?

Soma

Quote:

---

Originally Posted by phantomotap

O_o

Are you seriously still swallowing buzzword b......t?

If you don't think a project is a fit for your work, the opinions of other people are irrelevant.

Why would you even consider, again, evaluating a project based on marketing you didn't even try to investigate?

Soma

---

Fair point.

But phantom, without buzzwords on my resume, how will I ever land that dream programming job? XD

All joking aside, what's the point of Node.js? Like, it doesn't seem as powerful as Apache is so why do people hype it up so much? I knew I was encountering problems when I was shifting my development time to creating a type of routing instead of coding my actual site. But I figured it would be good to at least try something new/different.

So yeah, what's all the hubbub about Node.js when Apache actually does useful web stuff? I hear people talk about Node.js for web apps and all I can think is, why would anyone ever want to use this?

Edit :

Okay, I've done a bit more research. Apparently, Node is supposed to be great for web apps and not as a web server. Okay, cool. I can still use Apache. Awesome.

But how do I call a Node-based web app with Apache? Every thing I google seems to be using Node to first create a small web server before anything else is done. Is that what I have to do? Do I have to manage the httpd service and also create a bunch of Node processes that sit and listen for requests from pages served by Apache?

Oh man, I'm so lost and confused... Oh jeez. I will persevere though!

I've done something similar where I have a PHP page send a socket request to a Java server which does some cool stuff but I have to do all this extra Linux CLI stuff where I have to background the process and then disown it. Would I really have to do all this with my Node.js processes?

Quote:

---

Originally Posted by MutantJohn

But how do I call a Node-based web app with Apache? Every thing I google seems to be using Node to first create a small web server before anything else is done. Is that what I have to do? Do I have to manage the httpd service and also create a bunch of Node processes that sit and listen for requests from pages served by Apache?

---

That is the standard way of doing that kind of thing, yes. (For example, nginx + PHP works that way) But is that so bad? It looks easy..

Quote:

---

All joking aside, what's the point of Node.js?

---

O_o

I wasn't joking. I am not joking.

You need to stop being so willing to swallowing whatever buzzword b......t you find.

Quote:

---

Like, it doesn't seem as powerful as Apache is so why do people hype it up so much?

---

How do you not understand? The relative "power" is irrelevant.

The only people trying to hype "Node.js" are trying to sell you on the "Node.js" project.

You only need to stop listening.

Quote:

---

So yeah, what's all the hubbub about Node.js when Apache actually does useful web stuff?

---

Apples are food. Why do people eat oranges?

Quote:

---

I hear people talk about Node.js for web apps and all I can think is, why would anyone ever want to use this?

---

I like the idea of sharing code between client and server. I like the idea of being able to use extremely familiar tools to implement DOM manipulation in the browser and producing the consumed markup on the server with the same language. The "Node.js" project has some interesting and useful natural qualities, yet I have no intention of using the "Node.js" project beyond ensuring my library code works in the "Node.js" environment. I recognize the potential of the offering of the "Node.js" project, and you should also recognize the what is offered by the "Node.js" project. You should even consider evaluating a project based on the merits of the project, but you should not jump to a different stack just because you recognize the effort. You should certainly not jump to a different stack because hype.

[Edit]
I'm going to go off here on a bit of a tangent. Do you remember the last large Javascript thread you started? I was asked, as a consequence of the discussing that thread, to spend some time with an open implementation of exactly the sort of project which I repeatedly referenced. The code was a fairly basic "ReactJS" implementation. I asked some question then promptly refused any further involvement with the project. No. I didn't see anything disgusting. The group had just relatively recently decided to rewrite the earlier "AngularJS" code using the "ReactJS" framework. Why? I was given a few reasons, but the reality was hype. A coder rag had recently published, relative to the decision to rewrite, an article on how "ReactJS" was the best most refreshing drink ever. A group of coders threw away several weeks of work because of popularity contest sponsored by people who have no idea what framework might be most suitable for use by strangers.
[/Edit]

I prefer to use "lighttpd" as a server and Python as the language because I find the stack more suitable for my projects. I don't like "Apache" or the PHP language. I also know, in a poorly-remembered-never-really-very-good fashion, the Perl language. I flatly refuse to use Perl and PHP for new projects. (I also don't use Ruby or Go languages despite or maybe because of having used both in nontrivial projects.) I don't like "Nginx" even for static content. I would certainly recommend "lighttpd" and Python over "Node.js" for newcomers.

What does my preferences have to do with you? I have very good reasons for avoiding the PHP language, but the problems I have clearly don't bother you. (My reason for avoiding "Apache" is an arbitrary feeling of bloat.) From your various posts, the "Apache" and PHP stack you are using seems likely to be most appropriate for you; the fact alone is sufficient reason for you to ignore the "Node.js" hype.

Soma

That actually feels really good to hear, phantom.

Phew.

I think I should focus on making a good product instead of making a product with whatever's popular. Yeah, that sounds like it's more up my alley.

Edit:

Quote:

---

I wasn't joking. I am not joking.

---

Actually, I was talking about myself. I meant that this was a joke :

Quote:

---

But phantom, without buzzwords on my resume, how will I ever land that dream programming job? XD

---

Recruiters are typically dumb and will only consider people who have the right buzzwords on their resumes. With all the hate it gets, PHP might as well be a mark on my resume that says, please don't hire me.

I don't really get the hate, to be honest. Bad programmers are bad programmer no matter what language they choose to program in. PHP is very intuitive to me and reading the documentation to understand functions isn't that hard.

Quote:

---

Actually, I was talking about myself.

---

O_o

Fair enough; I misinterpreted as responding to a joke with a joke.

Quote:

---

Recruiters are typically dumb and will only consider people who have the right buzzwords on their resumes.

---

I don't agree with "recruiters are typically dumb" statement, but I wish I could refute the buzzword resume statement.

Quote:

---

I don't really get the hate, to be honest.

---

You remember making a reference to the complexity of a C++ example I posted?

If the only C++ examples a person had for reference was my code, would you understand the hate for the C++ language?

The sins and syntax of a language effect us all in different ways. I was aware of some of the problems of the PHP standard library before I ever started learning the language. I wrapped the relevant functions with my own code without ever really having a problem. My problem with PHP isn't related to quirks in the library. I can make C++ dance, but I can't make PHP dance. The PHP language has a tendency to stand between me and my solutions to problems. The quirks of the PHP language actually prevents me from writing the sort of code I want to write. If a tool repeatedly prevents you from working, you will be inclined to blame the tool.

Soma

Thank you, phantom. You really talked me off a ledge today.

Quote:

---

Originally Posted by Yarin

That is the standard way of doing that kind of thing, yes. (For example, nginx + PHP works that way) But is that so bad? It looks easy..

---

I will have to keep this in the back of my mind though. It could be very useful in the future.

Quote:

---

Originally Posted by phantomotap

I prefer to use "lighttpd" as a server and Python as the language

---

Without resorting to a VPS, you know of any good lightttpd or nginx based hosting companies out there?

Quote:

---

Without resorting to a VPS, you know of any good lightttpd or nginx based hosting companies out there?

---

O_o

I don't.

I also don't know, in case of followup, of any decently priced VPS at the moment.

Soma