Tag Archives: notes

This is a collection of my notes on Code School’s, Realtime Web with Node.js course. It follows the first set of notes on setting up a node server, using curl, read streams and write streams (levels 1-3), and the second set on using NPM, putting your dependencies in package.json and setting up the Express framework (levels 4-5).

Level 6

Socket.io allows you to create duplexed websocket connections in your node.js application. This allows data to be sent from client to server, from server to client or both simultaneously.

Setup

After installing socket.io with NPM, you need to require it on the server side.

// in your app.js
var socket = require('socket.io');
var app = express.createServer();
var io = socket.listen.app();

io.sockets.on('connection' function(client) {
  console.log('Client connection established...');
});

Then you need to include the library and do similar setup on the client side...

// inside your index.html (presumably loaded somewhere with express)
<script src="/socket.io/socket.io.js"></script>
<script>
  var server = io.connect('http://localhost:8080');
</script>

Sending Messages

You can send them either way. Here's an example from server to client:

// app.js
io.sockets.on('connection', function(client) {
  client.emit('messages', {greeting: 'Hello, client!'});
});

And to receive it:

//index.html
var server = io.connect(URL);
server.on('messages', function(data) {
  alert(data.greeting);  // will pop up saying, "Hello, client!"
});

Broadcasting to every client

One neat thing that socket.io can do that a traditional websocket can't do so easily is broadcast a message to every client.

client.broadcast.emit('messages', data);

Saving and retrieving data on the socket with get and set

client.set('location', 'Rome');
client.get('news', function(err, updates) {
  //do stuff with updates
});

Level 7

Redis is a key value store. It's much faster than a traditional relational database on big data sets but it doesn't store deep data structures, just keys and values. It offers strings, lists, sets, sorted sets and hashes, but no nested structures. Most importantly, Redis is non-blocking, so you your node app using redis won't be halted while retrieving data.

The Redis website has a fantastic interactive reference where you can try out the commands as if you were in a REPL. It's so good that there's little need to take notes on any of the commands.

Setting up Redis

I would have added to Code School's instructions a bit lacking here. You need a few installs to start using Redis comfortably--

  • redis - the library itself
  • redis-server - the server that you have to start for your node program to be able to access the db
  • redis-cli - the command line interface so you can inspect the database and experiment while figuring out how to make your node app do what you want it to do

Get them all:

npm install -g redis
npm install -g redis-server
npm install -g redis-cli

Basic usage examples

var redis = require('redis');
var redisClient = redis.createClient();

redisClient.set('PI', 3.14159265);
redisClient.get('PI');   // -&gt; 3.14159265

redisClient.lpush('World Wonder', 'Great Pyramid of Giza');
redisClient.lpush('World Wonder', 'Hanging Gardens of Babylon');
redisClient.lpush('World Wonder', 'Statue of Zeus at Olympia');
redisClient.lpush('World Wonder', 'Temple of Artemis at Ephesus');
redisClient.lpush('World Wonder', 'Mausoleum of Maussollos at Halicarnassus');
redisClient.lpush('World Wonder', 'Colossus of Rhodes');
redisClient.lpush('World Wonder', 'Lighthouse of Alexandria');

redisClient.lindex('World Wonder', 6);  // -> 'Colossus of Rhodes'
redisClient.lrange('World Wonder', 2,4);  // 2nd, 3rd and 4th elements, ->
//['Hanging Gardens of Babylon', 'Statue of Zeus at Olympia',  'Temple of Artemis at Ephesus']

Since there are only a few data types and the querying options aren't so complex, learning Redis is pretty simple compared to something like MySQL or mongoDB.

This is a collection of my notes on Code School's, Realtime Web with Node.js course. I posted notes on levels 1-3, yesterday. They covered setting up a node server, using curl, read streams and write streams.

Level 4

You can import external modules by using require('somemodule');. Exporting is done by setting the variable exports to some function. You can also set properties of exports to functions. Here's an example that shows both:

// "awesomod.js"
var awesomod = function(str) {
  return str.replace(/okay/, 'awesome');
}

var yawp = function() { 
  console.log("Awesome!");
} 

module.exports = awesomod;
module.exports.yawp = yawp;
// "index.js"
var awesomod = require("./awesomod");

s = "I'm okay.  How are you?";
console.log(awesomod(s));
awesomod.yawp();

Since we exported the function awesomod directly, that is what is called on line 5. It replaces the word "okay" with "awesome". The yawp function was attached onto the module with module.exports.yawp = yawp;, so it's found at awesomod.yawp

Using NPM

The Node Package Manager is great! It works just like apt-get or ruby gem. It goes out onto the internet and fetches the stuff you want to install. The basic syntax is

npm install somemodule

Afterwards you can access that package inside your javascripts with require('somemodule');, as shown above. Note that when there's no ./ in front of the module name, node will look for that module inside the default directory where NPM installs them.

To install programs you can run from the command line, such as coffee script, for example, use global -g flag.

nmp install coffee-script -g
coffee someapp.coffee

Setting up dependencies

By stating what dependencies your node app needs, you can use a single NPM command and it will go out and get all the modules on your dependencies list and the dependencies for those modules. Dependencies are specified by adding an object to a file package.json in your apps root directory. Here's an example of a basic package.json:

{
  "name": "The Okay App",
  "version": "1.0.0",
  "dependencies": {
    "express": ">=2.0.0",
    "redis": "~0.7",
    "ejs": "~0.7.1"
  }
}

There are a lot of options for setting up your dependencies.
= exactly that version
> higher than that version (Dangerous since the API could change!)
>= same or higher than that version (Dangerous!)
~0.7 at least 0.7 but less than 1.0 (Dangerous!)
~0.7.1 at least 0.7.1 but less than 0.8.0

After this package.json file is in place, just run one command to get them all:

npm install

Level 5

Level 5 was all about using Express.js, and for me it was the best part of the entire course. Node.js is very low level, which is great for making file uploading tools or simple chat programs or backend pieces of huge systems that need to run asynchronously... but it's a ton of work to do stuff like say making a database-powered website. The 15 minute "build a blog" demo that made such a big splash for rails would be a pain to do in Node.js directly. Express solves a lot of this pain. It's built on top of Node.js and provides help with routing, a template system for making views and a few other things. It certainly doesn't do all the things rails does for you, but it does a lot. From what I understand, it's more similar to the Sinatra framework.

Start the server and serve index.html:

var express = require('express');
var app = express.express.createServer();
app.get('/', function(request, response) {
	// __dirname is the current directory
	response.sendFile(__dirname + "index.html");
});
app.listen(8080);

Route Parameters:

app.get('/users/:name', function(req, response) {
	var name = req.params.name;
	// do stuff with the :name we read it...
});

Templates

Express Templates are .ejs files and they work just like the .erb files in rails -- Most of it is HTML, but the stuff between <% tags %> is executed as javascript and javascript between <%= %> tags gets printed on the screen. The blocks between these tags has access to parameters passed in from the server. In this example an object with names and friends is passed to the template:

// app.js
//...
response.render('users.ejs', {name: name, friends.fr});
//...
// users.ejs
<h1>Welcome, <% name %></h2>
<ul>
	<% friends.forEach(function(fr){ %>
	<li><%= fr.name %></li>
	<% }); %>
</ul>

This render a page for each users/1 that greets user#1 by name and has user#1's friends listed in an unordered list with one item generated with a name on it for each friend. At users/2 a page will be rendered that greets user#2 and has user#2's friends and so on for every user the app has stored.

Since moving to San Francisco, I've realized that despite its attractive market for start-up investments, Beijing is something of a tech backwater outside of a few companies. Not only are people here using a lot of technologies such as Ruby on Rails that are almost unused in China, but they're also crazy about a quite a few things I'd never even heard of while working at a fairly hip mobile start-up in Beijing. Top on that list was Node.js, so I worked through the free Node Beginner Book and then took Code School's course on it. Their interactive system is great for learning, but I still find the need to have some review notes to help it sink in. Here goes:

Level 1

Curl is useful! Why didn't I think of that? After setting up your Hello World server with this in your server.js:

var http = require("http");

http.createServer(function(request, response) {
  response.writeHead(200, {"Content-Type": "text/plain"});
  response.write("Hello World");
  response.end();
}).listen(8080);

There's no need to bother firing up a browser. You can verify the server's running with curl!

curl http://localhost:8080

Level 2

You can add a second listener for requests if you want to:

server = http.createServer(function (request, response) {
  // do stuff
}).listen(8080);

server.on('request', function(request, response){
  // 2nd listener
});

Since http.Server is an event emitter and http.createServer takes a requestListener as a parameter and requestListeners are functions that automatically listens to 'request' events, these two are identical:

http.createServer(function(request, response){ ... });
// is the same as
var server = http.createServer();
server.on('request', function(request, response){ ... });

The second syntax above is how we usually add our own event listeners in node.

Level 3

Read streams and write streams are event emitters. Request is a read stream; response is a write stream. You can make them do stuff with

somestring.on('someevent', function(data){...});

File Copying

var fs = require('fs');

var file = fs.createReadStream("somefile");
var newfile = fs.createWriteStream("somefile_copy");
// Pipe handles read/write speed mismatches for you
file.pipe(newFile); 

Handling back pressure manually if you don't use pipe

Unfortunately, there's the risk that the stream that's writing data out can't keep up with data that you're reading in. If memory fills up and more data keeps streaming in, bad things happen.

We can check to see if a writeStream's buffer is full with var bufferokay = writeStream.write(chunk);. If that returns false, you can shut down the readStream with readStream.pause(). Then once its buffer is empty writeStream will automatically send a 'drain' event. By listening for it and using a callback to readStream.resume() we can make sure the stream starts reading again once we're ready.

var fs = require('fs');

var file = fs.createReadStream("HUGEFILE.avi");
var newFile = fs.createWriteStream("HUGEFILE_copy.avi");

file.on('data', function(chunk) {
  var bufferokay = newFile.write(chunk);
  if (!bufferokay) { file.pause()}
  newFile.write(chunk);
});

newFile.on('drain', function(){
  file.resume();
})

file.on('end', function() {
  newFile.end();
});