Node.js Get Started : Async Programming, Callbacks, Variable Scopes

I started using Node.js in one of my projects and found it just awesome. In this blog i will write about basics of Node.js programming and how it works, and give a perspective of a PHP Programmer.

What is Node.js

As per the official website
“Node.js is a platform built on Chrome’s JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.”

What this means is :
1. Node.js runs on Chromo’s V8 engine. You write javascript code to run application and it runs from command line.
2. It is highly useful for network application, where you want to create your custom server. Its very easy to run a custom server using node.js including socket.io.
3. It is based on event driven model, as in a event is fired and you write a javascript function to run when that event gets fired. There are event for server connection, mysql query completion, file read finish, etc etc.
4. Its non-blocking: The entire programming model async (like an ajax request). Once an operation finishes, a function is called on completion. This is unlike languages like PHP where next line of code is only executed when previous line finished.

When to use Node.js

As a PHP Programming i will list of down few applications of Node.js which a language like PHP is very bad for.
1. Data intensive application, e.g you need to process millions on rows from a mysql/mongodb data source and send email reports. If you want to do this fast, with a lower memory foot print Node.js is best for it.
2. Custom Server Applications: like a chat server, multi player game engine, distribute applications.
3. Running Parallel Process: Because of its non-blocking model its very easy to run multiple processes in parallel.
4. Data Scrapping: Node.js is idea for data scrapping engines. With its ease of running parallel processes, able to execute server side javascript (JQuery), you can easily scrap data.
5. Server Side Applications using Javascript.

Creating Websites, Pages: Languages like PHP, Ruby are best for this purpose. Although there are libraries like Express, AngularJS to handle this in node.js, I still believe node.js is best for building applications not websites or UI.

What does async programming mean?

Firstly installing node.js is quite easy. Just visit their homepage for the guide.

Before starting Node.js programming, you need to understand two main concepts
1. Async Code Architecture
2. Event Driven Applications

Lets see what does Async Architecture Means

Below is a code of very basic Node.js App

[code]
function hello(){
console.log(‘end! really?’)
}
console.log(‘starting’);
fs.readFile(‘/home/myfile.txt’, function (err, data) {
if (err) throw err;
console.log(‘Data From File: ‘ + data);
});
hello();
[/code]

If a PHP Programming looks at this, the output should be like
1. starting
2. Data From File: ….
3. end! really?

But in Node.js it would be
1. starting
2. end! really?
3. Data From File: …

as you can see, the function hello() doesn’t wait for readFile to finish. Rather, the log statement in readFile is called once the entire file is read. If you have implemented ajax, this is exactly how ajax works as well.

This is how entire applications are build in Node.js, rather than assuming next live of code will be executed after current line finishes, you need to call a function when the code finishes.

Here is an outline of a sample application to read data from mysql and update it if need.

[code]

//start mysql connection
createConnection(err,function(data){
if(err){throw err;}
//mysql connection complete
fetchRows(err,function(err,data){
if(err){throw err;}
loop [data as row]
if(row.to_update == 1){
updateRow(err,function(err,res){
});
}
})
});
[/code]
As you can see entire above code is written in way such that, when on operation finish the next operation is called. We throw/handle an error if its there or else do the next operation.

What does event driven mean?

In Node.js applications when you typically create a server or connect to a server like mysql,mongo there are many events fired which you listen to and write your code. I will straight away give examples to show

Example of connecting to mongodb
[code]
var mongoose = require(‘mongoose’);
mongoose.connect(‘mongodb://127.0.0.1/mydb’);
var db = mongoose.connection;
db.on(‘error’, function callback() {
//on error event, do an operation
});
db.once(‘open’, function callback() {
//on open event do an operation
});
[/code]

Example of socket io server
[code]
var io = require(‘socket.io’).listen(80);
io.sockets.on(‘connection’, function (socket) {
//on connection event
socket.emit(‘news’, { hello: ‘world’ }); // fire your own custom event

socket.on(‘my other event’, function (data) {
// on event
});
});
[/code]

As you can see from above code, events are fired and operation are done as callbacks for those events.
Functions like “on” and “once” are used to listen to events.
Functions like “emit” are used to fire your own events.

Callback Functions

For a PHP programmer, to code in Node.js he needs to change the way he thinks. He should start thinking callbacks rather than classes, and async rather than sync 🙂

So how to code callbacks.
In the below example, we will create an array and then insert it into a mysql table. I am using this library https://github.com/felixge/node-mysql for mysql connection
[code]
var data = [];
for(var i=0;i<10;i++){
data.push({name:’Test Name’ + i});
}
//code to generate any array

var mysql = require(‘mysql’);
var connection = mysql.createConnection({
host : ‘localhost’,
user : ‘me’,
password : ‘secret’
});

connection.connect();

var status = 0;
for(i=0;i<data.length;i++){
insertData(data[i],function(err,result){
//anonymous callback function
status++;
if(status == data.length){
console.log(‘all insert operations completed, can exit now’);
connection.end(function(){
process.exit();
});
}
});
}

function insertData(row,callback){
var sql = "insert into table set ?";
sql.format(sql,{id:0,name:row.name});
connection.query(sql, function(err) {
if(callback){
callback(err);
// calling the callback function
}
});
}
[/code]
As you can see above we passed an anonymous function as a parameter to a function and hence were able to find out when the entire operation was completed. This anonymous function is usually called a callback function.
This is very useful trick in async programming.
Also its a convention in node.js, for the first variable to be err (error) and second to be result.

Anonymous Function and Variable Scope

Next trick we are going to see is also a very useful in node.js

Below is an example code

[code]
var data = [];
for(var i=0;i<10;i++){
data.push({name:’Test Name’ + i,id: Math.rand(0,1000)});
}
//code to generate any array

for(i=0;i<data.length;i++){
var id = data[i].id;
var name = data[i].name;
//check if file exists
fs.exists(‘/home/’ + name,function(exists){
if(exists){
console.log(id + "File Exists");
}
});
}
[/code]
Basically what above code is does is, to check if a file exist and log the file id.
But due to async nature of the operation, this above code will give incorrect result. The problem is that, inside the callback function we are using the ‘id’ variable which has been defined outside the function. The function fs.exists is async in nature, the id variable keeps getting updated and different id value might be logged in the exists callback. The fix this issue, we need to create an anonymous function and put the variable in scope. Here is the correct way to do it.
[code]
var data = [];
for(var i=0;i<10;i++){
data.push({name:’Test Name’ + i,id: Math.rand(0,1000)});
}
//code to generate any array

for(i=0;i<data.length;i++){
(function(row) {
var id = row.id;
var name = row.name;
//check if file exists
fs.exists(‘/home/’ + name,function(exists){
if(exists){
console.log(id + "File Exists");
}
});

})(data[i]);
}
[/code]
Basically in the above code, by creating an anonymous function, the variable id will be allocated in memory each time and won’t be overwritten like previous code. This is also a very important trick in Node.js

In this blog i wrote about basics of Node.js programming, its architecture.