Whether you’re new to SEO and looking to familiarise yourself with the topic or a seasoned developer looking for a fresh perspective, this blog is just what you are looking for.
- The important elements of JS you need to know
- What is Ajax?
- What is the Document Object Model (DOM)?
- What should SEOs look out for?
- Internal linking
- Content accessibility
- Single Page Application considerations
- Internal Linking
- Content accessibility
- Indexable URLs
- Descriptive metadata
- Using your browser’s “Inspect” feature
- URL Inspection tool in GSC
- Improve the page loading speed
- URL Inspection Tool
- Mobile-Friendly Test and Lighthouse
- Page Speed Insights
- Site: Command
- Chrome DevTools
Modern web pages are comprised of three major components:
- HTML – Hypertext Markup Language. Imagine this to be the “skeleton” to the web pages “physical form” – let’s use a Zebra as an example. As the bones in a Zebra skeleton provide structure, so too does HTML to a site. It organises the static content on a site into headings, paragraphs, lists etc.
- CSS – Cascading Style Sheets. Picture this as the “fur” that covers the Zebra skeleton. Much like a Zebra has an instantly recognisable black-and-white appearance, the CSS is the design, style and wow factor added to a website, making up the presentation layer of the page.
Put simply, as muscles make a Zebra run, JS is a programming language that makes webpages interactive and animated, bringing a page to life to engage a user.
- The ES4 version was meant to be a collaborative version between the two companies however Microsoft dropped out resulting in the version being skipped.
- By 2009 an agreement was made to combine all relevant work between all parties to help drive the language forward collaboratively, which is when we saw the ES5 release in the same year.
The important elements of JS you need to know
Google crawls a website using its ‘Googlebot’ crawler and indexes with the ‘Caffeine’ infrastructure. Each performs very different functions – Googlebot is all about discovery.
It’s a complex system, factoring in crawl queuing, scheduling and budget, URL importance, and server response time to name a few. But in essence, it finds all URLs and crawls them, discovering content and code present on a URL.
When the crawler finds URLs that have been changed or added since its previous visit, they are sent to the indexer. The indexer will try to make sense of the URL by analysing the content as well as assessing subprocesses like canonicalisation, page layout and more.
What is Ajax?
Asynchronous means “not existing or occurring at the same time”, so the web applications can communicate with a server without interfering with other functions or lines of code on a page by operating in the background.
Usually, all the assets on a page are requested and fetched from the server and then rendered on the page. AJAX can be implemented to update the content or a layout without initiating a full page refresh. Usefully, as often, pages on a site don’t differ greatly, using AJAX means only the assets that differ need to be loaded, which improves the UX.
A great example of this in action is Google Maps; the page updates as the user navigates without a full page reload.
What is the Document Object Model (DOM)?
Think of the Document Object Model (DOM) as the web browser’s actions taken after receiving the HTML document to render the page.
It is essentially an application programming interface (API) for markup and structured data such as HTML and XML.
From an SEO perspective, it’s important to understand the DOM, because it’s how Google analyses and understands webpages. Visually, the DOM is what you see when you “Inspect Element” in a browser.
The established approach, server-side rendering, involves a browser or a search engine bot (crawler) receiving HTML that describes the page exactly. So, the content is already in place, and your browser or search engine bot must download the attached assets (CSS, images, etc) to present how the page has been designed. As this is the traditional approach, search engines generally do not have a problem with server-side rendered content. Traditionally websites that render like this would be programmed in PHP, ASP or Ruby and might have used popular content management systems like Kentico, WordPress or Magento.
As shown in Google’s diagram, Googlebot places pages within a queue for their crawling and rendering processes. From here, Googlebot fetches a URL from the crawling queue and reads the robots.txt file to see whether the URL has been disallowed or not.
As Google is running two separate waves of indexing, it is possible for some details to be missed during the indexing process. For example, if you’re not server-side rendering crucial title tags and meta descriptions, Google may miss this on the second way, having negative implications on your organic visibility on the SERPs.
As previously mentioned, the main issue with JS is if bots are unable to find URLs and understand the site’s architecture. Crawling and indexing can become slow and inefficient.
Importantly this can lead to issues, whereby key pages are deemed unimportant due to a lack of internal link value. Likewise, relatively unimportant pages are misinterpreted as high value because there are plain HTML links pointing to them.
Further, because internal linking is a strong signal to search engines regarding the site’s architecture and the importance of pages, pages are crawled and rendered according to their perceived importance. You could, therefore, see Google investing more time crawling and rendering the wrong pages over the pages you actually want to rank.
The best way to resolve this issue is by providing search engines access to the resources they need by working with the website development team to determine which files should and should not be accessible to search engines.
One recommendation is pushState, currently supported by Google. It’s Navigation-based and used to manipulate the browser History API.
Simply, pushState updates the address bar URL, ensuring only what needs to change on the page is updated.
The best use is seen with infinite scroll (when the user scrolls to new parts of the page the URL will update). This means the user shouldn’t need to refresh the page, as the content updates as they scroll down while the URL is updated in the address bar.
What should SEOs look out for?
A number of search engines will deploy headless browsing, a type of software that can access web pages, but does not show the user and transfers the content of the web page to another program that runs on the backend. A headless browser helps to render the DOM to gain a better understanding of the user’s experience and the content situations on the page.
Above the fold refers to the part of the web page that is visible when the page initially loads. The subsequent portion of the page that requires scrolling is called “below the fold”. This can be apparent across a range of devices, including; desktops, mobiles, iPads and many more. In order to remove this, please refer to the section below that outlines key tools to use.
Single Page Application considerations
A single-page application (SPA) is a web application/ website that has been primarily designed and built to operate efficiently on the web. These pages are dynamically rewritten and loaded with the pieces you require, as opposed to loading an entire page from the server.
The SPA approach provides a fast loading time, uses less bandwidth and provides the user with a pleasant experience by making the application behave more like a desktop application. It should be noted that there are many different SPA framework options available, depending on the use of the application. These include; React.js, Angular.js, Backbone.js, Preact, Next.js and hundreds more.
When rendering SPAs, John Mueller stated the following.
It’s not always perfect, and certainly not easy, but for some sites it can work well, even if you rely on client-side-rendering (just JS, no server-side-rendering). YMMV 🙂— I am John – ⭐ ⭐ SRS BZNS ONLY ⭐ ⭐ (@JohnMu) July 16, 2018
There can be many hurdles when Google attempts to crawl and index the content on the SPA. Therefore, if you’re using SPAs, it’s recommended to test out multiple times, using the Fetch command, to understand what Google is able to pick up.
This sounds scary on its own but if the majority of your business comes in digitally, then this sort of issue will result in a drastic drop in business and conversions until resolved. Whilst we’re primarily talking about organic search within this article, if the page’s content can’t be seen by the search engines then it could also prevent any paid advertisements via a search engine from being shown too, as the search engine will see it as having no relevance to your target keyword and show other advertisers that do have relevance as the priority instead.
Any advertisements like dynamic search ads (DSA) campaigns that utilise content from the page for the advertisement won’t display due to finding no content.
If all the HTML and content are loaded in after the initial server response (client-side rendering) then it can potentially result in the rendering process not detecting any content on the page if the rendering process test finishes before the content has loaded in.
Pages require indexable URLs that offer server-side support for each landing page. This includes each category, subcategory and product page.
With each individual page on the site having a specific focus and target, they should also include descriptive titles and meta descriptions to help search engine bots and users precisely detect what the page is about. Not only this, but it helps users determine if this is the most suitable page for their search query.
Using your browser’s “Inspect” feature
Once the rendered HTML has been obtained and meets the level of a traditional landing page expected by Google, many impacting factors will solve themselves.
You can test whether your website uses client-side rendering by going to view-source and comparing against inspect, if all HTML and content can be found in both view-source and inspect then the website is utilising server-side rendering if you can’t find any of the content of the pages within view-source then it is utilising client-side rendering. There is the potential that it may be a mix as well where important HTML and content are being sent via server-side rendering but a more dynamic section of the page is being delivered via client-side rendering. You can utilise Diffchecker for a more comprehensive view of what is different between view-source and Inspect.
An example of this can be seen on YouTube, we have a screenshot of a video on the website’s homepage that looks to deliver personal recommendations based on your viewing history:
But when we looked at the source code and ran a search for “your writing process”, it didn’t find anything:
But should we then run this same search on Inspect, we can see the video title, which indicates that the website uses client-side rendering and that this content is dynamically loaded in after the initial page load:
URL Inspection tool in GSC
The URL inspection tool allows you to analyse a specific URL on your website to understand the full status of how Google is viewing it. The URL inspection tool provides valuable data around the crawling, indexing and further information from Google’s index, such as successful indexation or structured data errors causing issues.
Improve the page loading speed
The main thing to consider when looking to provide a personalised experience for visitors is that Google and other search engines are not a fan of ‘cloaking’ which is where the main content is different between a crawler like Googlebot and a user.
When personalising content one should look at only modifying highly visible items that don’t change the main purpose or content of the page. Highly visible items are things like headers, introductions or recommended/similar items.
Other items that you could look to personalise are imagery on the page, this could be based upon a visitor’s geo-location or if they’re logged into the site, personal preferences set on their account, an example of this could be to use more suitable imagery for a user that is using a dark mode preference.
Personalisation can end up slowing the website down if it has to run checks before delivering potentially different elements or imagery. To avoid the major pitfalls in personalisation and speed performance decreases it’s best to avoid any third-party plugins or rules for personalisation and if possible utilise a CDN to help deliver larger elements such as imagery whilst mitigating any speed performance decreases.
You may be asking yourself, is it worth it? This seems like a lot of work/consideration for some minor changes to page(s). Well yes, there is going to be a fair bit of work involved and determining the rules that you will utilise and continuous testing to see how the personalisations perform, but the potential these personalisations could have on a visitor converting are extremely high.
URL Inspection Tool
Found within Google Search Console, the URL Inspection Tool shows information on whether Google was able to crawl / index a page, whether any errors are occurring and why.
Mobile-Friendly Test and Lighthouse
Google’s mobile-friendly tester provides information regarding how easy it is for a visitor to navigate your website on a mobile device and any issues with it meeting requirements to be classified as ‘Mobile Friendly’, however, this tool is being removed in December 2023.
For a more detailed description of any mobile-friendly issues or recommendations for the page, you can utilise the Lighthouse test found in Google Chrome’s DevTools which we get to via Inspect and then navigate to the Lighthouse tab. Once a test has been completed, navigate down to the mobile-friendly section which will have its own overall score out of 100 and any recommendations to improve mobile friendliness.
Page Speed Insights
Google’s Page Speed Insights tool (PSI) effectively details the mobile and desktop performance of a page. In addition to this, this tool also provides recommendations on how this can be improved.
If you are going down this list and have tried out Lighthouse you will notice that PSI and Lighthouse are similar, this is because PSI utilises Lighthouse for its test.
Lighthouse is more commonly used by developers as it provides in-depth recommendations and analyses of the page you’re currently on, whilst PSI will fetch the page and test it. This means that if you have a page in staging/testing that is not yet live and behind a login, then PSI will likely be unable to test the page whilst Lighthouse can, provided the user is past the login stage.
Page Speed Insights also provides CWV data on how the page has performed on average with users if there are enough visits and data available from user experiences to provide this information.
The site: command is one of the most direct tools to help see if Google has properly indexed your content. In order to do this, you can complete the following command in a Google search:
site: https://www.example.com “text snippet or query”
Replace https://www.example.com with your website or a specific category/page if you want to see pages that contain that category or a specific page only.
The text snippet or query is optional, if you don’t know the URL of a specific page you want to test, but know its overall topic/focus you can enter a strong relevance keyword for that page for it to find the most relevant results found on your site for said keyword.
The two can be used in combination, so you know the page is within a specific category and you don’t know the URL off by heart, but you know it’s about a specific topic, then:
site: https://www.example.com/category/ “specific topic”
If no “text snippet or query” is added to the site command and you enter the root domain URL (often the homepage) then the Google search result will provide a rough count of how many pages on the site are indexed on the search engine.
Diffchecker is a unique tool that allows you to compare two types of text files and review the differences between both. This is especially useful for performing an analysis of a webpage’s original source code against the rendered code. This tool delivers comprehensive comparisons of how the content has changed after being rendered.
Chrome DevTools is a set of tools for experienced web developers to build directly into Google’s Chrome browser. Chrome DevTools can help you edit and make quick styled changes without needing to use a text editor. It helps to discover problems in a fast manner, which in turn, can help to build better websites in a quicker period.
You can access this via Inspect which we talked about earlier or via the 3 dot menu > More Tools > Developer Tools
Screaming Frog: A crawler that is constantly updated with new features, has lots of helpful usage guides on their site, allows for custom extraction of specific elements on the site and is used by a very large percentage of SEOs within the industry.
SiteBulb: A crawler that is growing in popularity and provides easy-to-understand ‘hints’ which are recommendations on areas of improvement for the site’s SEO/visibility on search engines.
BuiltWith is a free website profiler that helps to discover what framework websites are being built with. Not only this, but it’ll also tell you about any integrations with a third-party website that the website has.