Handling an AJAX response in JavaScript (with or without jQuery)

The Problem & the solution

One of the most common questions asked on StackOverflow tends to revolve around AJAX, and the inability to immediately utilize the response of an AJAX request as follows;

var response = '';
var xhr = new XMLHttpRequest(); // Usual mix-and-matching for x-browser omitted for brevity

xhr.onreadystatechange = function () {
  response = this.responseText;
}
xhr.open('GET', '/ajax.php', true);
xhr.send(null);

// Try and use 'response' or 'xhr.responseText' here (note: this will not work!)

Or equally in jQuery:

var response = '';
var xhr = jQuery.ajax('/ajax.php', function (result) {
  response = result;
});

// try and use response here (note: this will not work!)

So common is this question on StackOverflow, that it features in the JavaScript tag description. The answer to each of these questions always advises the developer to use a callback to handle the response. We’ll look at why a callback is required shortly, but for those of you in a rush, I will first show the solution;

var xhr = new XMLHttpRequest(); // Usual mix-and-matching for x-browser omitted for brevity
// no need to declare 'response' here

xhr.onreadystatechange = function () {
  if (this.readyState == 4 && this.status == 200) {
    var response = this.responseText;

    // use response in here.
  }
}

xhr.open('GET', '/ajax.php', true);
xhr.send(null);

And the jQuery equivalent;

jQuery.get('/ajax.php', function (response) {
  // use response here; jQuery passes it as the first parameter
});

The explanation

A callback is required because an AJAX, as the name Asynchronous JavaScript And XML suggests, is asynchronous. When you initiate a AJAX request using either the native XMLHttpRequest method or via jQuery, the HTTP request is sent, but the JavaScript engine does not wait for a response. Instead, the flow of execution continues. To enable us to monitor the progress of the HTTP request, we are allowed to provide callbacks which are executed when the state of the HTTP request changes. A callback can be provided to a native XMLHttpRequest object via the onreadystatechange attribute;

var xhr = new XMLHttpRequest();

xhr.onreadystatechange = function () {
  // Code inside here is executed each time the progress of the HTTP request advances.
  // The current state can be retrieved via `this.readyState`, which returns a value ranging
  // from 0 to 4 (inclusive).

  if (this.readyState == 4) { // If the HTTP request has completed 
    if (this.status == 200) { // If the HTTP response code is 200 (e.g. successful)
      var response = this.responseText; // Retrieve the response text          
    };
  };
};

xhr.open('GET', '/ajax.php', 'true');
xhr.send(null);

When using jQuery, callbacks are specified differently depending on which AJAX method you used; methods such as jQuery.get, jQuery.post, jQuery.getJSON accept only a success callback by providing a function as a parameter. jQuery.ajax however, permits multiple, separate callbacks to be specified as name:value pairs in the object passed in the last parameter.

jQuery.get('/ajax.php', function (response) {
    // Handle success here
});

jQuery.ajax('/ajax.php', {
    beforeSend: function () { /* AJAX request is about to be sent */ },
    complete: function () { /* AJAX request has completed */},
    success: function () { /* AJAX request has completed successfully */},      
    error: function () { /* AJAX request has completed with errors */}
})

The introduction of callbacks leaves the flow of execution looking something like this;

var xhr = new XMLHttpRequest(); // Usual mix-and-matching for x-browser omitted for brevity

// Point 1

xhr.onreadystatechange = function () {
  // Point 4

  if (this.readyState == 4) {
    // Point 5

    if (this.status == 200) {
      var response = this.responseText;

      // Point 6
    }
  }
}

// Point 2

xhr.open('GET', '/ajax.php', true);
xhr.send(null);

// Point 3

Point 1, 2 and 3 will be executed once in ascending order immediately. The execution of point 4 will then wait until some time later when the state of the HTTP request progresses. Point 4 will be reached and executed multiple times until the HTTP request reaches the completion state (readyState == 4), upon which the flow of execution will reach Point 5. Point 6 will be executed if the HTTP request completed successfully. The same flow of execution can be witnessed in the following jQuery code;

// Point 1

jQuery.get('/ajax.php', function (result) {
    // Point 6
});

// Point 3

It is a common pattern to show some sort of progress indicator at Point 3; usually something as simple as a loading spinner, to indicate that an asynchronous HTTP request is in progress. It would be ideal to provide updates to the status at Point 4, however the values observed through readyState are not useful enough for this. In point 5, the HTTP request is finished, so the loading spinner can be removed, and either a success or error message can be shown, depending on whether the status is a successful HTTP error code.

JSON vs JSONP

Why we need JSONP (JSON‘s shortcomings)

JSON is a language designed for lightweight data exchange in an format that is easy for both humans and computers to understand.

Unfortunately, as much as JSON appears to be the best thing since sliced bread as far as web developers are concerned, the technologies used by JavaScript to send and receive JSON messages (XMLHttpRequest, and Microsoft’s family of ActiveXObject‘s) are restricted by the same origin policy which is enforced in all major browsers; i.e. you cannot make a request to a resource using a different protocol, domain or port to which the current page is using.

However, it was discovered that the JSON language we know and love could work on a technology which is not restricted by the same origin policy with only a small amount of changes required. The combination of this technology, along with the adapted form of JSON was branded JSONP.

An introduction to the new technology

You’ll be disappointed to hear that the new-and-exciting technology that is fundamental to JSONP is nothing more than the bog-standard <script> tag.

The full power of the <script> tag is often overlooked, or at the very least, taken for granted:

  1. <script> tags are not restricted by any same origin policy, so can make requests to any domain.

  2. <script> tags can be added programatically, which allows developers to request resources via the <script> tag as and when required:

    var element = document.createElement("script"); // create the element 
    element.src = 'http://somewhere.com/some/resource.js'; // set the target resource
    document.getElementsByTagName("head")[0].appendChild(element); // add the element to the DOM
    
  3. The src attribute of the <script> tag is not restricted to just JavaScript files. Whatever the target, it will be retrieved via the relevant transport protocol (HTTP GET/ HTTPS GET), and the content will be evaluated by the JavaScript engine. If the content is not valid JavaScript, a Parse Error will be thrown.

We can combine these observations to form a two way communication channel between the web page and the remote resource.

  1. Using observations 1 and 2, the client can initiate requests to remote resources. To make the requests useful, it is usual for the target to be server scripted page; such as a PHP file.

        // Example utility function
        function requestViaScript(url) {
          var element = document.createElement("script");
    
          element.src = url;
    
          document.getElementsByTagName("head")[0].appendChild(element);
        }
    
        // Example usage
        requestViaScript('http://example.com/endpoint.php?param1=a&param2=b');
    
  2. Observation 3 states that the resource’s contents is evaluated by the JavaScript engine. Therefore, if the resource returns valid JavaScript which has an effect (e.g. calls a function), the server has a method to respond to the request.

How JSON fits in

As touched upon earlier, JSON is attractive to JavaScript developers because it’s syntax is valid JavaScript; when evaluated, it becomes either an object or an array. However, a JSON response does not have any effect; as the resultant array/object isn’t assigned to anything.

However, we can easily modify the response so instead of returning JSON like so:

{ "foo": [1,2,3,4] }

It is returned like this:

someFunction( { "foo": [1,2,3,4] } );

Which provides the effect we need (someFunction is executed), and JSONP (i.e JSON with padding), is born.

The final piece of the jigsaw

The last question that needs answering is “How does the server know the name of the function to call?”; and the answer is simple.

We pass the name of the function as a parameter in the request.

This parameter is commonly named the “callback” parameter.

requestViaScript('http://example.com/endpoint.php?param1=a&param2=b&callback=foo'); // Server knows to surround the JSON response with foo()

That is all there is to JSONP. Below is some code which provides a complete example of how a simple message exchange is performed on both the client and the server, to help tie any loose ends. That aside, I hope you’ve found this article useful.

Worked Example

In this simple ficticious example, the webpage (client) will sent a number to the server. The server will add VAT to the number, and return the new value.

JavaScript:

function requestViaScript(url) {
  var element = document.createElement("script");

  element.src = url;

  document.getElementsByTagName("head")[0].appendChild(element);
}

function calculateVAT(price) {
  requestViaScript('http://example.com/endpoint.php?price=' + price + '&callback=handleResponse');
}

function handleResponse(response) {
  alert("The price " + response.price + " is " + response.vat + " including VAT");
}

PHP (http://example.com/endpoint.php)

if (isset($_GET['price']) && isset($_GET['callback'])) {
  $response = array();
  $response['price'] = $_GET['price'];
  $response['vat'] = $_GET['price'] * 1.20;

  echo $_GET['callback'] . '(' . json_encode($response) . ')';
}

JSON: What it is, and what it isn’t

What JSON is

JSON is a data interchange format similar to languages such as XML and YAML. Its language independence allows programing languages to communicate with each other, though the process of stringification and parsing.

The popularity of JSON can be put down to its simplicity; it is easy for both humans and computers to read, and its language encompasses most of the types which are common across modern web languages. Furthermore, it is considerably more lightweight than other exchange formats such as XML. Variations of JSON such as BSON reduce the size of a JSON representation even further.

The syntax of the language bears a strong resemblance to the JavaScript programming language; this is no coincidence as the language was suggested by Douglas Crockford, a name known to many due to his contributions to the JavaScript language. The standardised version of the language (RFC 4627) supports the following types:

  • Number
  • Boolean
  • Null
  • String (characters captured with double quotes)
  • Ordered lists (comma separated types captured by “[“ and “]”)
  • Unordered sets (comma separated string:type pairs captured by “{“ and “}”)

You can easily check if a string is valid JSON by entering its contents on the jsonlint.com website. For more detailed explanations of the valid JSON types, see the json.org website.

What JSON isn’t

It is all too common for developers to get confused as to what JSON is; particularly amongst the circles of web developers. This confusion can no doubt be put down to JSON’s similarity to JavaScript.

JSON is nothing more than a data format. Once the JSON formatted string has been received by the programming language, it is usually parsed immediately into a language dependant structure; such as an array or an object. Likewise in the reverse, something is only “JSON” when it has been stringified from a language dependant type (such as an array or an object) into a valid JSON string.

This concept is so often misunderstood that I shall try an analogy to enforce the point; imagine you are purchasing a piece of flat pack furniture, but the only one left in stock is the display model, which you agree to purchase. The display model is dissembled from a usable product and packaged in a box (read: stringification). The packaged box is now in a format equivalent to JSON; it is unusable in every sense, but provided the sender has a postbox (read: stringifer), and the receiver a letter box (read: parser), it can be handled. Upon being received, the product has to be assembled (read: parsed) for it to become usable again.

Another common misconception is that JSON is AJAX or vice versa. AJAX is a buzz word for a group of technologies available within browsers which allow a developer to make HTTP(s) requests on demand/ asynchronously. AJAX does not specify (nor does it care) what format the HTTP(s) response is in; JSON is just one available option, along with HTML, XML and JSONP.

Continuing with the posting-flat-pack-furniture theme, AJAX can be likened to the cardboard packaging used to transport products; it does not care whether it ships a bed (read: JSON), a table (read: XML) or a wardrobe (read: HTML).

Lastly, the syntax shared between JavaScript and JSON often leads to some confusion as to what JSON is. The syntax of JSON is subset of JavaScript syntax; meaning that all valid JSON is valid JavaScript, but not vice versa. For instance, JSON does not support the types undefined, function or Date. Another common mistake is to believe that values surrounded by single quotes ('foo') is valid JSON string; however a JSON string must be surrounded by double quotes ("foo"). Additionally, whilst the object literal syntax in JavaScript is similar to that of JSON’s unordered set, it should be noted that the key to a JSON unordered set must be a string, whilst JavaScript’s object literal syntax accepts either a number, or the toString() version of any other type.