Improving Page Speed with Incremental Loading

2019-04-27

Summary: you can use django-include-by-ajax to improve the performance and usability of your website by forcing some parts of the Django website page to be loaded and shown before other parts of the page.

Web browsers load and render traditional HTML pages from top to down, from left to right and as a developer you have little control over what will be shown first, second, and last. However, sometimes you need a different loading sequence to improve user experience and usability. Let's examine a couple of cases when it is advantageous to have primary content showing up immediately and secondary content loading in a moment.

Case 1. Above the Fold vs. Below the Fold

People want speed. 47% of visitors expect the website to be loaded in less than 2 seconds. If the website takes more than 3 seconds to show up, it's a big chance, that you will lose 40% of visitors. If you sell something on your website, every one-second delay causes 7% fewer visitors becoming buyers.

One technique to improve the perception of the speed of the website is to display the visible part of the screen as soon as possible, and then load the rest of the website in another go. Usually, the website pages are long vertically scrollable areas. The part of it that fits in the screen is called "above the fold" and the part underneath is called "below the fold".

Primary content above the fold and secondary content below the fold

It is recommended to load the part above the fold in 6 requests, including all your HTML, CSS, JavaScript, images and fonts. It's only 6 requests for a reason - that's the maximal number of requests that most browsers keep to the same HTTP/1.1 server at the same time. With HTTP/2 there is no such limitation.

You can only achieve this minimal load if you bundle and minimize your CSS and JavaScript to single files, and use only a couple of images and fonts. Going one step further you can split your CSS and JavaScript into parts that are used above the fold, and the ones that are used below the fold.

Case 2. Main Content vs. Navigation

For the users to have best user experience and smooth loading, you could display the content of articles or blog post first, and then load and display the website navigation in the header, sidebars, or footer.

Content is primary and the navigation is secondary

If the visitor navigated to a specific page of your website, they most likely want to see the content of that page rather than navigate out to other pages.

If you have extensive nested navigation, you can also save some milliseconds of its loading at the first request, by skipping it there, but loading it by Ajax at the next go.

Additionally, if visitor disables JavaScript in their browser, they will still be able to read the content.

Case 3. Own Content vs. Third-party Content

Wouldn't you agree, websites that show ads before their own content are pretty annoying? One way to improve the user experience is to show the main content at first and show the ads or third-party widgets after several seconds.

Own content is primary and third-party widgets are secondary

The primary content will be correctly indexed by search engines, whereas the included widgets might be skipped, depending on implementation, which we'll examine next.

Solution 1. Iframes

One way to load the delayed secondary content is to use iframes.

Pros:

Works without JavaScript.

Cons:

For each iframed section, you need a separate HTML with custom CSS.
You have to predefine and hardcode all heights of each secondary section, so it wouldn't work well with increased or decreased font size or different amounts of content.
You cannot have interactive elements like tooltips that would go outside the boundaries of the iframe.

Solution 2. API Calls by Ajax

The page would load with empty placeholders for the secondary content and then some JavaScript function would load content for the missing sections in HTML, JSON, or XML format by Ajax, parse them, and include into the placeholders. This approach has been used by Facebook.

Pros:

You can use the same global CSS for everything.
The amount of content is flexible, so the designs would look good with different variations.

Cons:

For each secondary section, you need to define a separate API endpoint.
There are many extra requests (unless you use GraphQL for that).

Solution 3. A Second Request to the Same Page with Specific Query Parameters

The page loads with empty placeholders for the secondary content. A JavaScript function uses Ajax to load the HTML of the same page this time containing all rendered primary and secondary content. Then another JavaScript function goes through all placeholders and fills the content from the second load.

Pros:

You can use the same global CSS for everything.
The amount of content is flexible, so the designs could look good with different variations.
Each page uses a single data endpoint.
Only one extra request is necessary for the full HTML.

Cons:

If there is a lot of primary content and not so much of secondary content, it might take too long to load and parse the secondary content.

Implementation for a Django Website using django-include-by-ajax

You can implement the third solution in a Django website using my open-source Django app django-include-by-ajax. It is meant to be understandable and simple to use for frontend Django developers, who don't touch Python code but need to work on the layouts and styling.

The idea is that instead of including different sections of a template with the {% include template_name %} template tag, you do the same using {% include_by_ajax template_name %} template tag. This template tag renders as an empty placeholder unless you access the page from a search crawler or if you access the page with a specific query parameter. Otherwise, it works more-or-less the same as the {% include %} template tag.

By adding jQuery and one jQuery-based JavaScript file to your page template, you enable the magic that does all the loading and parsing. Since version 1.0, CSS and JavaScript files can also be included in those delayed sections.

You can see django-include-by-ajax in action at the start page of my personal project 1st things 1st. There I use the above-the-fold case with the visible content coming to the screen almost immediately and the offscreen content loading in several more seconds.

Installation

It should be trivial to convert any standard heavy website page to a page loading the secondary content dynamically. There are mainly these steps to follow:

1. Install the app

Install the app with your Python package manager:

(venv)$ pip install django-include-by-ajax==1.0.0

2. Edit Django project settings

Put the app into the INSTALLED_APPS in your Django project settings:

# settings.py
INSTALLED_APPS = [
    # ...
    # Third-party apps
    'include_by_ajax',
    # ...
]

3. Modify the base template

Put these in your base.html:

{% load staticfiles %}
<script src="https://code.jquery.com/jquery-3.4.0.min.js" crossorigin="anonymous"></script>
<script src="{% static 'include_by_ajax/js/include_by_ajax.min.js' %}" defer></script>

It should also work with older or newer jQuery versions.

4. Include the template tag

Use the new template tag in any template where you need it:

{% load include_by_ajax_tags %}
{% include_by_ajax "blog/includes/latest_blog_posts.html" %}

You can even define the content for the placeholder that will be shown while the main content is loading:

{% load include_by_ajax_tags %}
{% include_by_ajax "blog/includes/latest_blog_posts.html" placeholder_template_name="utils/loading.html" %}

5. Handle JavaScript event

If you need some JavaScript action to be called after all content is loaded, you can use the custom include_by_ajax_all_loaded event on the document like this:

$(document).on('include_by_ajax_all_loaded', function() {
    console.log('Now all placeholders are loaded and replaced with content. Hurray!');
});

I would be glad if you tried it on some of your projects and see how it improved user experience and loading times. If you use it in production, please mention it in the comments of this blog post.