Login Register

Storing Offline Data

An offline application is useless if it does not have access to data. Dojo Offline provides two ways to store data, depending on your needs. The first is Dojo Storage, which provides a simple, persistent hash table abstraction; and Dojo SQL, which provides SQL based access to your data.

Dojo Storage

Dojo Storage contains a very simple hash table abstraction, providing methods such as put() and get(), which you can use to quickly store data without having to delve into SQL. Under the covers it saves this data into the Google Gears persistent storage system.

Saving data is easy. You can store simple strings:

dojox.storage.put("someKey", "someValue");

or more complicated JavaScript objects:

var car = {type: "Nissan", color: "white", price: 20000, optional:"air-conditioner, stereo"};
dojox.storage.put("complexKey", car);

JavaScript objects are automatically saved as they are. Note, however, that browser objects will not get serialized, such as references to the DOM or an XMLHttpRequest object.

Also note that key names can only be letters, numbers, and the under score character. Spaces are not allowed.

Loading data is just as easy; if you stored a JavaScript object, it will be returned to you as a JavaScript object:

var value = dojox.storage.get("someKey");
  var car = dojox.storage.get("complexKey");

If the object is not found, null is returned.

Further info and advanced usage on Dojo Storage can be found here.

Dojo SQL

Dojo Storage can be great, but some times you need the full power of a relational data store. Enter Dojo SQL.

Dojo SQL is an easy to use, thin layer over Google Gear's relational storage layer. Executing SQL is easy:

var results = dojox.sql("SELECT * FROM DOCUMENTS");

Results are returned as ordinary JavaScript objects, unlike Google Gears, which makes working with SQL much easier. For example, if you have created a table named DOCUMENTS:

dojox.sql("CREATE TABLE IF NOT EXISTS DOCUMENTS ("
			+ "fileName		TEXT NOT NULL PRIMARY KEY UNIQUE, "
			+ "content		TEXT NOT NULL) ");

that has five rows in it (i.e. five documents), and you call

dojox.sql("SELECT *
FROM DOCUMENTS")
, an array will be returned that has five rows, one for each document. Dojo SQL creates an object for each row, automatically taking each of the column names, such as fileName and content, and using those as the object literals:
var results = dojox.sql("SELECT * FROM DOCUMENTS");

// document 1
alert("The first documents file name is " + results[0].fileName + " and it's content is " + results[0].content);

// document 2
alert("The second documents file name is " + results[1].fileName + " and it's content is " + results[1].content);

Inserting data is just as easy:

dojox.sql("INSERT INTO DOCUMENTS (fileName, content) VALUES (?, ?)",fileName,
contents);

Simply put a question mark for each parameter in the SQL, and then provide the actual variables that will fill them in as variable length arguments at the end. You can use this for any SQL statement where you want to provide a parameter:

dojox.sql("SELECT * FROM DOCUMENTS WHERE fileName = ?", someFileName);

For a full list of SQL statements that can be executed by SQLite, see here. The SQL to do full-text indexing can be found here.

Dojo SQL automatically handles opening and closing the database; for advanced usage information on how to manually do this yourself, which you normally shouldn't need to do, see here.

Encrypting Offline Data

A common issue with offline web applications is that they must download data to be stored on the local machine. If the machine is a laptop, sensitive data could be stored that would be at risk if the laptop is stolen. For example, imagine you are dealing with an application that is downloading student records with social security numbers; you don't want to have these stored in the clear if the laptop is stolen.

To address this use-case, Dojo SQL includes a cool feature that makes it easy to encrypt and decrypt specific columns of data. For example, imagine we have a CUSTOMERS table with three columns, a last name, a first name, and a social security number:

dojox.sql("CREATE TABLE CUSTOMERS ("
					+ "last_name TEXT, "
					+ "first_name TEXT, "
					+ "social_security TEXT"
				+ ")"
	);

For the first and last names, we don't care if they are stored in the clear. However, for the social security column we would like to encrypt it.

Dojo SQL adds two new SQL keywords that make this easy, ENCRYPT() and DECRYPT(). Let's see how to use them.

To encrypt the SS number, we would do the following:

var password = "foobar";
dojox.sql("INSERT INTO CUSTOMERS VALUES (?, ?, ENCRYPT(?))", "Neuberg", "Brad", "555-34-8962",
		password,
		function(results, error, errorMsg){
			if(error){ alert(errorMsg); return; }
		});

In the example above, we provide our three INSERT parameters as usual; however, for the last one, the social security number, we put the ENCRYPT(?) keyword around it. After providing these as variable arguments at the end, we provide a password. The password is used for encryption rather than a key -- it is passed into the underlying cryptographic system; you should not store this password as a cookie or into Dojo Storage or Dojo SQL. Instead, the user should be prompted to enter it when they start using your application. If you store it then a laptop thief could simply use it to unlock the local data store.

The final argument is a callback. If you use the ENCRYPT or DECRYPT keywords then the call to dojox.sql becomes asychronous. This is because under the covers we spawn Google Gears Worker Pool threads to do the actual cryptography in such a way that the browser doesn't screech to a halt. When they are done the callback will be called. If are doing decryption, as you will see below, then the results will have a normal Dojo SQL JavaScript results object, but with the values decrypted. If there was an error then error will be true and errorMsg will have a reason for the error.

Decryption is just as simple; simply put the DECRYPT keyword around the result columns you want decrypted:

var password = "foobar";
dojox.sql("SELECT last_name, first_name, DECRYPT(social_security) FROM CUSTOMERS",
		password,
		function(results, error, errorMsg){
			if(error){ alert(errorMsg); return; }

			// go through decrypted results
			alert("First customer's info: " 
					+ results[0].first_name + " "
					+ results[0].last_name ", " 
					+ results[0].social_security);
		});

In this example we simply print out the decrypted results for the first row. If the password is wrong then the decrypted results will still be decrypted; there is no way to detect an incorrect password attempt programatically.

You can combine several columns at once into a single ENCRYPT statement, such as ENCRYPT(?, ?, ?, ?), and you can do the same with DECRYPT statements, such as DECRYPT(first_name, last_name, social_security).

Under the covers Dojo SQL's cryptography is powered by 256-bit AES, using the passphrase you provide to derive the key. Specifically, we use the JavaScript AES implementation given here if you would like to study how it works; special thanks to Chris Veness for contributing the AES encryption code to Dojo.

Loading Offline Data On Page Load

Whether you use Dojo Storage or Dojo SQL, there is an important scenario you need to think about. If the user loads your application while offline, you must initialize your application using your stored data rather than making a network call. You should do this after Dojo Offline has loaded:

initialize: function(){
	// initialize our UI using offline data if necessary
	var documents = null;	
        if (dojox.off.isOnline){
		// make a network call to get our data
		// ...
	}else{ // we are offline
		documents = dojox.storage.get("documents");
        }
}

In the example above, our initialize() method is called when Dojo Offline is done loading. Our UI must then get a list of documents to make available; if we are online (dojox.off.isOnline), then we just make a network call. If we are offline, then we just load our documents from offline storage, in this case Dojo Storage.

We cover how to get your downloaded data for offline use in the next section, covering syncing.