How to dynamically (via AJAX) add new items to a bound list model, in ASP MVC.NET

Imagine you have a form, which allows a user to add n entries. In my case, a user was creating a Building and was defining each Room. Each room had a “Name” and “Area”;

public class Building
{
    [Required]
    public string Name { get; set; }
    public List<Room> Rooms { get; set; }

    public Building()
    {
        Rooms = new List<Room>();
    }
}

public class Room
{
    [Required]
    public string Name { get; set; }
    [Range(1,200)]
    public int Area { get; set; }
}

I had EditorTemplates defined for both Building and Room;

// Views\Building\EditorTemplates\Building.cshtml
@model DynamicListBinding.Models.Building

<div class="form-group">
    @Html.LabelFor(x => x.Name)
    @Html.TextBoxFor(x => x.Name, new { @class = "form-control" })
    @Html.ValidationMessageFor(x => x.Name)
</div>

// Views\Building\EditorTemplates\Room.cshtml
@model DynamicListBinding.Models.Room

<div class="panel panel-default">
    <div class="panel-body">
        <div class="form-group">
            @Html.LabelFor(x => x.Name)
            @Html.TextBoxFor(x => x.Name, new { @class = "form-control" })
            @Html.ValidationMessageFor(x => x.Name)
        </div>

        <div class="form-group">
            @Html.LabelFor(x => x.Area)
            @Html.TextBoxFor(x => x.Area, new { @class = "form-control" })
            @Html.ValidationMessageFor(x => x.Area)
        </div>
    </div>
</div>

… and my Create.cshtml view looked like this;

// Views\Building\Create.cshtml
@model DynamicListBinding.Models.Building
@{
    ViewBag.Title = "Create";
}

@using (Html.BeginForm())
{ 
    <h2>Create</h2>

    <h3>Building</h3>
    @Html.EditorFor(x => x)

    <h3>Rooms</h3>
    @Html.EditorFor(x => x.Rooms)

    <input type="submit" />
}

In fact, you can download the skeleton of this solution from here (or just browse the repository on GitHub here).

Now, I didn’t know how many rooms each building had. My options were:

  1. Give the user an ample amount (say 20) of “Room” entries on the page to start off with, and hope the user wasn’t creating a mansion or castle.
  2. Load additional “Room” entries on-demand using AJAX.

If you’re wanting to use #1, unfortunately you’re in the wrong place, as this article explains how to go about #2. Sorry about that.

What makes #2 hard is how the DefaultModelBinder requires my list of rooms to be named (very specifically), like so;

<input type="text" name="Rooms[0].Name" />
<input type="number" name="Rooms[0].Area" />

<input type="text" name="Rooms[1].Name" />
<input type="number" name="Rooms[1].Area" />

<input type="text" name="Rooms[2].Name" />
<input type="number" name="Rooms[2].Area" />

There are 2 main problems with this:

  1. Imagine I allow my users to delete Rooms as well, and let’s say they delete Rooms[1]. The HTML becomes like so;
    <input type="text" name="Rooms[0].Name" />
    <input type="number" name="Rooms[0].Area" />
    
    <input type="text" name="Rooms[2].Name" />
    <input type="number" name="Rooms[2].Area" />
    

    Because of how DefaultModelBinder works, Room[2], will disappear from my model upon submission, as DefaultModelBinder requires index’s to be consecutive, and stops when it reaches a non-existent index.

  2. When loading additional fields in via AJAX, I need to be able to tell my endpoint what the next index is (because DefaultModelBinder requires index’s to be consecutive).

With these issues in mind, it’s very hard to allow your list of entries to be dynamically added and, potentially, deleted, whilst keeping these indexes sequential.

To make this less hard, Microsoft allow you to provide a .Index field (they just don’t tell anyone this…), which allows you to use any index you want; it doesn’t have to be sequential, and hell, it doesn’t even have to be a number.

We’d then be able to allow our users to delete fields, and the following HTML submission would now work;

<input type="hidden" name="Rooms.Index" value="0" />
<input type="text" name="Rooms[0].Name" />
<input type="number" name="Rooms[0].Area" />

<input type="hidden" name="Rooms.Index" value="2" />
<input type="text" name="Rooms[2].Name" />
<input type="number" name="Rooms[2].Area" />

However, our AJAX endpoint for adding new fields still needs to have some idea which index’s have been used, so it doesn’t generate additional fields with the same name. This approach also introduces the difficulty of using @Html.EditorFor(), whilst being able to output the .Index field.

To save the day, enter a HtmlHelper extension, EditorForMany();

using System;
using System.Collections.Generic;
using System.Linq.Expressions;
using System.Text;
using System.Web.Mvc;
using System.Web.Mvc.Html;

public static class HtmlHelperExtensions
{
    /// <summary>
    /// Generates a GUID-based editor template, rather than the index-based template generated by Html.EditorFor()
    /// </summary>
    /// <typeparam name="TModel"></typeparam>
    /// <typeparam name="TValue"></typeparam>
    /// <param name="html"></param>
    /// <param name="propertyExpression">An expression which points to the property on the model you wish to generate the editor for</param>
    /// <param name="indexResolverExpression">An expression which points to the property on the model which holds the GUID index (optional, but required to make Validation* methods to work on post-back)</param>
    /// <param name="includeIndexField">
    /// True if you want this helper to render the hidden &lt;input /&gt; for you (default). False if you do not want this behaviour, and are instead going to call Html.EditorForManyIndexField() within the Editor view. 
    /// The latter behaviour is desired in situations where the Editor is being rendered inside lists or tables, where the &lt;input /&gt; would be invalid.
    /// </param>
    /// <returns>Generated HTML</returns>
    public static MvcHtmlString EditorForMany<TModel, TValue>(this HtmlHelper<TModel> html, Expression<Func<TModel, IEnumerable<TValue>>> propertyExpression, Expression<Func<TValue, string>> indexResolverExpression = null, bool includeIndexField = true) where TModel : class
    {
        var items = propertyExpression.Compile()(html.ViewData.Model);
        var htmlBuilder = new StringBuilder();
        var htmlFieldName = ExpressionHelper.GetExpressionText(propertyExpression);
        var htmlFieldNameWithPrefix = html.ViewData.TemplateInfo.GetFullHtmlFieldName(htmlFieldName);
        Func<TValue, string> indexResolver = null;

        if (indexResolverExpression == null)
        {
            indexResolver = x => null;
        }
        else
        {
            indexResolver = indexResolverExpression.Compile();
        }
                        
        foreach (var item in items)
        {
            var dummy = new { Item = item };
            var guid = indexResolver(item);
            var memberExp = Expression.MakeMemberAccess(Expression.Constant(dummy), dummy.GetType().GetProperty("Item"));
            var singleItemExp = Expression.Lambda<Func<TModel, TValue>>(memberExp, propertyExpression.Parameters);

            if (String.IsNullOrEmpty(guid))
            {
                guid = Guid.NewGuid().ToString();
            }
            else
            {
                guid = html.AttributeEncode(guid);
            }

            if (includeIndexField)
            {
                htmlBuilder.Append(_EditorForManyIndexField<TValue>(htmlFieldNameWithPrefix, guid, indexResolverExpression));
            }

            htmlBuilder.Append(html.EditorFor(singleItemExp, null, String.Format("{0}[{1}]", htmlFieldName, guid)));
        }

        return new MvcHtmlString(htmlBuilder.ToString());
    }

    /// <summary>
    /// Used to manually generate the hidden &lt;input /&gt;. To be used in conjunction with EditorForMany(), when "false" was passed for includeIndexField. 
    /// </summary>
    /// <typeparam name="TModel"></typeparam>
    /// <param name="html"></param>
    /// <param name="indexResolverExpression">An expression which points to the property on the model which holds the GUID index (optional, but required to make Validation* methods to work on post-back)</param>
    /// <returns>Generated HTML for hidden &lt;input /&gt;</returns>
    public static MvcHtmlString EditorForManyIndexField<TModel>(this HtmlHelper<TModel> html, Expression<Func<TModel, string>> indexResolverExpression = null)
    {
        var htmlPrefix = html.ViewData.TemplateInfo.HtmlFieldPrefix;
        var first = htmlPrefix.LastIndexOf('[');
        var last = htmlPrefix.IndexOf(']', first + 1);

        if (first == -1 || last == -1)
        {
            throw new InvalidOperationException("EditorForManyIndexField called when not in a EditorForMany context");
        }

        var htmlFieldNameWithPrefix = htmlPrefix.Substring(0, first);
        var guid = htmlPrefix.Substring(first + 1, last - first - 1);

        return _EditorForManyIndexField<TModel>(htmlFieldNameWithPrefix, guid, indexResolverExpression);
    }
        
    private static MvcHtmlString _EditorForManyIndexField<TModel>(string htmlFieldNameWithPrefix, string guid, Expression<Func<TModel, string>> indexResolverExpression)
    {
        var htmlBuilder = new StringBuilder();
        htmlBuilder.AppendFormat(@"<input type=""hidden"" name=""{0}.Index"" value=""{1}"" />", htmlFieldNameWithPrefix, guid);

        if (indexResolverExpression != null)
        {
            htmlBuilder.AppendFormat(@"<input type=""hidden"" name=""{0}[{1}].{2}"" value=""{1}"" />", htmlFieldNameWithPrefix, guid, ExpressionHelper.GetExpressionText(indexResolverExpression));
        }

        return new MvcHtmlString(htmlBuilder.ToString());
    }
}

Now, I’ll save how this works for another article; all we care about in this one is that it does. To use it, I have to do 2 things;

  1. Add a property to the model, which the EditorForMany helper will store the generated index in. Without this, the Html.Validation* methods will not work (see here for a deep-dive into “why” for the curious).
    public class Room
    {
        [Required]
        public string Name { get; set; }
        [Range(1,200)]
        public int Area { get; set; }
        public string Index { get; set; }
    }
  2. Substitute my Html.EditorFor(x => x.Rooms) in Create.cshtml with:
    @Html.EditorForMany(x => x.Rooms, x => x.Index);
    

… and all of our problems are solved! You’ll see that Html.EditorForMany() uses GUIDs rather than numbers for indexes. This removes the need for us to tell our AJAX endpoint which indexes as been used; as our AJAX endpoint will instead just generate a new GUID. Html.EditorForMany() also takes care of seamlessly producing the .Index field for us as well.

All that’s left to do is to get our AJAX endpoint up and running. To do this, I define a new action on my BuildingController;

[OutputCache(NoStore = true, Duration = 0, VaryByParam = "*")]
public ActionResult AddRoom()
{
    var building = new Building();
    building.Rooms.Add(new Room());

    return View(building);
}

… and create a view Views\Building\AddRoom.cshml;

@model DynamicListBinding.Models.Building
@{
    Layout = null;
}

@Html.EditorForMany(x => x.Rooms, x => x.Index)

… and lo-and-behold, after adding the necessary JavaScript to AJAX in a new Room entry at the click of a button (to see what exactly what I changed, see the diff on GitHub), my form now works like a dream.

Note that there is a second way to use Html.EditorForMany(); which is included for scenarios where the hidden <input /> generated by Html.EditorForMany() would be an invalid child of the HTML parent (e.g. within elements such as a <tbody> or <ul>). Here, you’d pass false as the 3rd parameter to Html.EditorForMany(), then within your editor view call Html.EditorForManyIndexField(x => x.Index) to include the hidden <input /> in a valid place in the DOM.

In closing, I’d first like to point you to a blog post on haaked.com, and an answer by DaveMorganTexas on Stack Overflow; which helped me massively on this. I’d also like to thank the people who have commented on this article to highlight issues and improvements which I’ve incorporated over time.

You can browse the GitHub repository of the code used in this article, and download the final code as a zip from here.

Tracking/ joining parallel AJAX requests with jQuery

When launching a number of AJAX requests simultaneously, it can often be painful to track the requests, and seemingly necessitates the introduction of counter variables to detect when all the AJAX requests have completed. However, since jQuery 1.5, the various AJAX methods have returned a Promise object (an object which exposes only the “safe” methods of a Deferred). This, coupled with the jQuery.when method, allows you to easily add handlers to handle either the successful completion of all AJAX requests, or the failure of one; without the need for you to track anything yourself! To use this approach, simply pass the Promise’s returned by your AJAX calls as arguments to jQuery.when(), then add handlers using the done(), fail() and always() to the Promise returned by jQuery.when(), as demonstrated in the example below.

jQuery.when(jQuery.get('/foo'), jQuery.post('/bar'), jQuery.get('/baz')).done(function (a, b, c) {
    $('#output_a').text(a[0]);
    $('#output_b').text(b[0]);
    $('#output_c').text(c[0]);
});

The ith argument passed to done() is an array of arguments that correspond to the response of the ith Promise passed to when() (i.e. the arguments are available in the order they were provided; the order the responses were received in make no difference). For the example above, this means that a, b and c are all arrays of [ data, textStatus, jqXhr ]. To get the textStatus of the second AJAX call we’d use b[1]. To get the jqXhr object of the 3rd AJAX call we’d use c[2]. All fail() handlers are invoked the first time one of the provided Promises fails. Here however, the arguments are provided as they would be passed into a fail() handler bound directly to the failed AJAX request; i.e. the first argument is the jqXHR object, the second is the textStatus and the third is the error thrown. The behaviour of always() changes depending on whether a request failed or not; if all requests succeed, always() behaves like done(). If a request fails, always() behaves like fail().

jQuery.when(jQuery.get('/foo'), jQuery.post('/bar'), jQuery.get('/baz')).done(function (a, b, c) {
    $('#output_a').text(a[0]);
    $('#output_b').text(b[0]);
    $('#output_c').text(c[0]);
}).fail(function (jqXhr, textStatus, error) {
    alert('A request failed with the error "' + jqXhr.status + '"');
}).always(function () {
    // because of the varying parameter behaviour, its difficult to
    // do anything which uses them.

    $('#loading_spinner ').hide();
});

The signature of jQuery.when() (e.g. having to pass each Promise as a separate argument) can be cumbersome, and make it seem impossible to use when the number of AJAX requests you have cannot be determined. However, we can easily use Function.apply(), and use jQuery.when() using an array of Promise‘s instead;

var ajaxRequests = [jQuery.get('/foo'), jQuery.post('/bar'), jQuery.get('/baz')];

jQuery.when.apply(jQuery, ajaxRequests).done(function () {
    for (var i=0;i<arguments.length;i++) {
        console.log('Response for request #' + (i + 1) + ' is ' + arguments[i][0]);
    }
});

If you have not come across the arguments object yet, you can read about it here. As can be seen in the example above, apply() returns the “master-Deferred” the same way using jQuery.when() directly does, allowing you to bind done(), fail() and always() handlers as before.


For more reading, check out the jQuery documentation for jQuery.when(), or the question on Stack Overflow that triggered me to write this blog post!

Submitting a form as an AJAX request using jQuery

It is often the case where you have a form in your HTML, and you want to submit either all or part of it via AJAX, rather than via standard HTTP GET/ POST.

You can easily do this in jQuery using the serialize() method, which, when called on a jQuery object containing either a form element or form controls (<input />‘s, <select>‘s etc.), will return a URL-encoded string.

Example HTML:

<form id="the-form">
    <input name="username" type="text" value="mattlunn" />
    <input name="password" type="password" value="pass1234" />
</form>

Example JavaScript:

alert($('#the-form').serialize()); // username=mattlunn&password=pass1234 

Try it yourself in this jsFiddle.

This URL-encoded string can be used directly as the GET query string, or as your POST body, so all that is left is to construct a jQuery AJAX request, passing this string as the content of the request:

jQuery.ajax({
    url: '/some/endpoint.php',
    method: 'GET',
    data: $('#the-form').serialize()
}).done(function (response) {
    // Do something with the response
}).fail(function () {
    // Whoops; show an error.
});

You might find this snippet of jQuery code a good starting point for implementing AJAX forms on your website. It copies the method and target from the <form /> control, and makes the AJAX request for you.

/**
 * Utility function to help submit forms/ form controls via AJAX.
 * @param selector: Optional. Parameter passed to find() to restrict which elements are serialized. Default is to return all elements
 * @param success: Required. A function to be executed when the AJAX request completes. Arguments are standard AJAX args. 'this' is the form.
 * @param error: Required. A function to be executed when the AJAX request fails. Arguments are standard AJAX args. 'this' is the form.
 */
jQuery.fn.ajaxify = function (selector, success, error) {
    // Handle the optional case of selector.
    if (arguments.length === 2) {
        error = success;
        success = selector;
        selector = undefined;
    }

    return this.on('submit', function (e) {
        // Copy the options from the '<form />' control.
        jQuery.ajax({
            url: this.action,
            method: this.method || 'GET',
            data: (typeof selector === 'undefined' ? $(this).serialize() : $(this).find(selector).serialize())
        }).done(jQuery.proxy(success, this)).fail(jQuery.proxy(error, this));

        e.preventDefault(); // Don't forget to stop the form from being submitted the "normal" way.
    });
};

The above function can be used like so;

jQuery(document).ready(function ($) {
    $('form').ajaxify(function (response) {
        // Handle a successful submission
    }, function (xhr, error) {
        // Handle an error'd submission
    })
});

If you’re interested in reading further, you may want to check out the definition for a “successful control”; as only these types of elements will be contained in the string returned by serialize(). Furthermore, you may also want to checkout the jQuery plugin “forms”, which is a plugin designed to bring unobtrusive AJAX to your forms.

Sending FormData with jQuery.ajax()

This post explains how you can send the HTML5 FormData object as an AJAX request with jQuery. If you’re simply looking to learn how you can submit a form via AJAX using jQuery please checkout my other blog post instead; Submitting a form as an AJAX request using jQuery.


HTML5 introduces FormData to allow developers to build forms objects dynamically (and can even include files from the user’s file system), and then to send this form object via AJAX. You can easily use FormData with an XMLHttpRequest by passing the FormData instance as the parameter to xhr.send(). However, what if you want to send the FormData using jQuery?

Unfortunately, it’s not as easy as setting the data attribute to the FormData instance; jQuery runs anything that isn’t a string through jQuery.param() to serialize the objects keys into key1=a&key2=b etc; running FormData through this doesn’t end too nicely.

However one of jQuery.ajax()‘s many options is processData, which allows you to turn off this behaviour by setting it’s value to false. So, to use FormData with jQuery.ajax() it’s as simple as:

var formData = new FormData($('form')[0]); // Create an arbitrary FormData instance

jQuery.ajax('/endpoint.php', {
    processData: false,
    contentType: false,
    data: formData
});

Note we’ve also had to set contentType to false as well; unless contentType is specified as an option to jQuery.ajax(), jQuery reverts to the default of "application/x-www-form-urlencoded". By setting contentType to false we prevent this option from being set, and the browser implementation of XMLHttpRequest (which jQuery uses behind the scenes of course) will set the correct Content-Type header for us.

Be aware that no amounts of jQuery will make FormData cross browser. Whilst most browsers have supported FormData since the stone age (Chrome 7, Firefox 4.0 and Safari 5), Internet Explorer and Opera are the noticeable exceptions, with support only being added in Internet Explorer 10 and Opera 12 respectively. As there’s no real polyfill/ fallback you can employ for FormData, you should use feature detection and degrade gracefully as appropriate.

if (!("FormData" in window)) {
    // FormData is not supported; degrade gracefully/ alert the user as appropiate
}

The risk of JSON hijacking

JSON hijacking is an exploit which has not had the publicity it perhaps deserves. It is a real risk to website security; as much as CSRF or XSS.

It’s easier to understand how JSON hijacking works if you have a basic understanding on the differences between JSON and JSONP.

Traditionally, to utilize a JSON response we need to make a request via XHR, retrieve the responseText, and then parse it using JSON.parse() (or similar). As XHR requests are restricted to the same origin policy (SOP), this is only relevant if we’re making the request to the same domain we’re on; and because we (usually) control that domain, there is little risk.

JSONP requests (made by inserting <script> tags) are not restricted by the SOP, but the problem here is that a JSON response normally has no effect; i.e. there’s no problems retrieving a JSON response via this method, but there’s no way to capture or utilize the response;

<script src="http://www.remote-server.com/get-json.php">
{
    "user_id": 1234,
}
</script>

However, clever folks discovered that in some browsers, you can cause an effect to happen;

  1. You can override the Array constructor to do something with the array

    Array = function () {
        for (var i=0;i<this.length;i++) {
            alert(this[i]);
        }
    }
    
  2. To access object attributes, you can define a setter to capture the setting of that attribute;

    Object.prototype.__defineSetter__("user_id", function (value) {
        alert("user_id is being set to: " + value);
    });
    

Bear in mind that neither of these exploits are cross browser. However, this is a real threat that has been used to target the likes of Google and Twitter.

So how exactly does this work?

  1. An attacker injects the above snippet(s) on a website you visit (a website he owns?)
  2. When the request for JSON from the 3rd party domain is made, any cookies you have for that domain are sent with the request; if you’ve got a session on that website, the website is non-the-wiser that it’s not actually you requesting that page.
  3. When the response is returned, the above snippets are executed, and the attacker can manipulate/ steal the response; any personal/ important details contained in there are now his.

Fortunately, there are a number of ways to fix this;

  1. Rigorously check for the X-Requested-With header to check the request came via XHR.
  2. Add breaking code to the JSON response.
    • Facebook lead the JSON response with a for (;;); (infinite loop).
    • Google lead the JSON response with a throw 1; (throw an error).

Because of these additions, when the browser evaluates the response, the object/ array declarations are never reached. Obviously in your code, you’ll need to strip the leading mush out before you attempt to parse the JSON;

var json = JSON.parse(this.responseText.slice("for (;;;);".length));

or

var json = JSON.parse(this.responseText.slice("throw 1;".length));

For further reading, you may be interested in the following information articles; JSON Hijacking on http://haacked.com, JSON Hijacking on http://thespanner.co.uk.