The Problem & the solution

One of the most common questions asked on StackOverflow tends to revolve around AJAX, and the inability to immediately utilize the response of an AJAX request as follows;

var response = '';
var xhr = new XMLHttpRequest(); // Usual mix-and-matching for x-browser omitted for brevity

xhr.onreadystatechange = function () {
  response = this.responseText;
}
xhr.open('GET', '/ajax.php', true);
xhr.send(null);

// Try and use 'response' or 'xhr.responseText' here (note: this will not work!)

Or equally in jQuery:

var response = '';
var xhr = jQuery.ajax('/ajax.php', function (result) {
  response = result;
});

// try and use response here (note: this will not work!)

So common is this question on StackOverflow, that it features in the JavaScript tag description. The answer to each of these questions always advises the developer to use a callback to handle the response. We’ll look at why a callback is required shortly, but for those of you in a rush, I will first show the solution;

var xhr = new XMLHttpRequest(); // Usual mix-and-matching for x-browser omitted for brevity
// no need to declare 'response' here

xhr.onreadystatechange = function () {
  if (this.readyState == 4 && this.status == 200) {
    var response = this.responseText;

    // use response in here.
  }
}

xhr.open('GET', '/ajax.php', true);
xhr.send(null);

And the jQuery equivalent;

jQuery.get('/ajax.php', function (response) {
  // use response here; jQuery passes it as the first parameter
});

The explanation

A callback is required because an AJAX, as the name Asynchronous JavaScript And XML suggests, is asynchronous. When you initiate a AJAX request using either the native XMLHttpRequest method or via jQuery, the HTTP request is sent, but the JavaScript engine does not wait for a response. Instead, the flow of execution continues. To enable us to monitor the progress of the HTTP request, we are allowed to provide callbacks which are executed when the state of the HTTP request changes. A callback can be provided to a native XMLHttpRequest object via the onreadystatechange attribute;

var xhr = new XMLHttpRequest();

xhr.onreadystatechange = function () {
  // Code inside here is executed each time the progress of the HTTP request advances.
  // The current state can be retrieved via `this.readyState`, which returns a value ranging
  // from 0 to 4 (inclusive).

  if (this.readyState == 4) { // If the HTTP request has completed 
    if (this.status == 200) { // If the HTTP response code is 200 (e.g. successful)
      var response = this.responseText; // Retrieve the response text          
    };
  };
};

xhr.open('GET', '/ajax.php', 'true');
xhr.send(null);

When using jQuery, callbacks are specified differently depending on which AJAX method you used; methods such as jQuery.get, jQuery.post, jQuery.getJSON accept only a success callback by providing a function as a parameter. jQuery.ajax however, permits multiple, separate callbacks to be specified as name:value pairs in the object passed in the last parameter.

jQuery.get('/ajax.php', function (response) {
    // Handle success here
});

jQuery.ajax('/ajax.php', {
    beforeSend: function () { /* AJAX request is about to be sent */ },
    complete: function () { /* AJAX request has completed */},
    success: function () { /* AJAX request has completed successfully */},      
    error: function () { /* AJAX request has completed with errors */}
})

The introduction of callbacks leaves the flow of execution looking something like this;

var xhr = new XMLHttpRequest(); // Usual mix-and-matching for x-browser omitted for brevity

// Point 1

xhr.onreadystatechange = function () {
  // Point 4

  if (this.readyState == 4) {
    // Point 5

    if (this.status == 200) {
      var response = this.responseText;

      // Point 6
    }
  }
}

// Point 2

xhr.open('GET', '/ajax.php', true);
xhr.send(null);

// Point 3

Point 1, 2 and 3 will be executed once in ascending order immediately. The execution of point 4 will then wait until some time later when the state of the HTTP request progresses. Point 4 will be reached and executed multiple times until the HTTP request reaches the completion state (readyState == 4), upon which the flow of execution will reach Point 5. Point 6 will be executed if the HTTP request completed successfully. The same flow of execution can be witnessed in the following jQuery code;

// Point 1

jQuery.get('/ajax.php', function (result) {
    // Point 6
});

// Point 3

It is a common pattern to show some sort of progress indicator at Point 3; usually something as simple as a loading spinner, to indicate that an asynchronous HTTP request is in progress. It would be ideal to provide updates to the status at Point 4, however the values observed through readyState are not useful enough for this. In point 5, the HTTP request is finished, so the loading spinner can be removed, and either a success or error message can be shown, depending on whether the status is a successful HTTP error code.