Tips SEO friendly Angular with Prerender, Docker and Nginx

Making a modern web framework based application SEO friendly is not a problem any more. Google proposed a specification for making web applications easier to parse by crawlers by pre-rendering the app views when requested by a crawler.

This tutorial will help you make an Angular based application SEO friendly using Docker container. You can use and apply this tutorial to any javascript frontend framework.

Using a Docker container is optional. It will help us save time setting up the prerender host and keep our server components well isolated. You can strip the Docker part and just run the host locally on your server.

Enable hashbang urls

Your web application needs to use hashbang urls so that search engines know how to ask for the SEO friendly pages.

Preparing the Prerender host

We will use docker-compose to quickly setup the prender host. I will assume docker and docker-compose are already installed on your host server.

Docker compose uses a simple configuration file to quickly pull images and setup docker containers. We will use the following docker-compose.yml

prerender:
    image: cerisier/prerender
    ports:
        - "4000:3000" 
    restart: always #auto start prerender after reboot.
#replace 4000 with the desired port for the prerender app

prerender:

image: cerisier/prerender

ports:

- "4000:3000"

restart: always #auto start prerender after reboot.

#replace 4000 with the desired port for the prerender app

Put this file on your host server and run the following command from the same working directory as the file:

docker-compose up -d

1	docker-compose up -d

You can check if the prerender host is running with the following command:

sudo netstat -tl | grep 4000

1	sudo netstat -tl \| grep 4000

and should see a process listenting on port 4000

Nginx configuration

Now that the prerender host is running and listening, we will setup our web application’s nginx configuration to forward requests from crawlers to the prerender host.

I will assume your already configured nginx to proxy requests to your web applications using the proxy_pass directive. This what the nginx configuration file for your virtual host should look like.

server {

        listen 80;
        server_name yourapp.com;
        root html;
        index index.html;

        access_log /var/log/nginx/yourapp.com/access.log;
        error_log /var/log/nginx/yourapp.com/error.log;

        location / {

                # Make sure to pass any headers through the proxy here and not inside the if statements !  
                proxy_set_header Host $host;
                proxy_set_header Accept-Encoding "";
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;

                # Defining a variable that will be used to process normal or crawler requests separately
                set $prerender 0; 

                if ($http_user_agent ~* "baiduspider|twitterbot|facebookexternalhit|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator") {
                    set $prerender 1;
                }

                if ($args ~ "_escaped_fragment_") {
                    set $prerender 1;
                }

                if ($http_user_agent ~ "Prerender") {
                    set $prerender 0;
                }

                if ($uri ~ "\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff)") {
                    set $prerender 0;
                }
                
                
         
                if ($prerender = 1) {
                    
                    # when a crawler asks for a page proxy the request to the prerender host
                    rewrite .* /$scheme://$host$request_uri? break;
                    proxy_pass http://localhost:4000;
                }


                if ($prerender = 0) {
                    # This is a normal requests that is proxied to your web app
                    proxy_pass http://127.0.0.1:8098;
                    break;
                }
        }

}

server {

listen 80;

server_name yourapp.com;

root html;

index index.html;

access_log /var/log/nginx/yourapp.com/access.log;

error_log /var/log/nginx/yourapp.com/error.log;

location / {

# Make sure to pass any headers through the proxy here and not inside the if statements !

proxy_set_header Host $host;

proxy_set_header Accept-Encoding "";

proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

proxy_set_header X-Forwarded-Proto $scheme;

# Defining a variable that will be used to process normal or crawler requests separately

set $prerender 0;

if ($http_user_agent ~* "baiduspider|twitterbot|facebookexternalhit|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator") {

set $prerender 1;

}

if ($args ~ "_escaped_fragment_") {

set $prerender 1;

}

if ($http_user_agent ~ "Prerender") {

set $prerender 0;

}

if ($uri ~ "\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff)") {

set $prerender 0;

}

if ($prerender = 1) {

# when a crawler asks for a page proxy the request to the prerender host

rewrite .* /$scheme://$host$request_uri? break;

proxy_pass http://localhost:4000;

}

if ($prerender = 0) {

# This is a normal requests that is proxied to your web app

proxy_pass http://127.0.0.1:8098;

break;

}

You are done

When a crawlers hit your server, nginx will forward the request to the prerender app. Prerender will automatically generate the page if it does not exist or just serve it from the cache. Otherwise the request is served from your web app.