How to Make Google Fall in Love with Your Website

Ever wondered why some websites seem to magically appear at the top of search results while others get lost in the digital abyss? The secret is technical SEO, and it’s time to crack the code.

What is Technical SEO?

Technical SEO involves strategically optimising a website’s technical elements to improve search engine crawling, indexing, and overall performance. According to Google’s Search Central documentation, it encompasses a comprehensive set of practices that ensure search engines can effectively understand, crawl, and index a website’s content. Moz, a leading SEO research platform, describes technical SEO as the foundation that allows search engines to access, crawl, interpret, and index a website without any issues. The core components include:

• Website structure optimisation
• Improving site speed and performance
• Ensuring mobile-friendliness
• Implementing secure protocols
• Creating clean, semantic HTML

Why Technical SEO Matters

Search engines favor sites with top-tier technical performance, fast-loading pages, mobile-friendliness, and accessibility all play a crucial role. If search engines can’t crawl your pages, your content won’t even make it to the rankings, no matter how brilliant it is. This can lead to lost traffic, frustrated users, and missed revenue opportunities. On the flip side, a seamless user experience signals to search engines that your site deserves the spotlight.

According to a study by Backlinko, websites that meet Google’s Core Web Vitals see up to 24% higher organic search visibility. They reward websites that demonstrate fast loading times, provide excellent user experiences, maintain robust security protocols, and feature clear, logical structural designs.

Research from SEMrush indicates that technical SEO directly impacts:

• Search engine rankings
• User engagement rates
• Conversion potential
• Overall website performance

By investing in technical SEO, businesses can significantly improve their search rankings, increase visibility, and drive more organic traffic. Google’s own research shows that improving site speed can reduce bounce rates and increase user satisfaction.

To understand technical SEO better, let’s break down two of its key processes: crawling and indexing.

What is Crawling and Indexing?

Crawling is the process where search engines send out bots, often called “spiders” or “crawlers,” to explore the pages on your website. These bots follow links, gather information about your content, and identify new or updated pages to understand what your site is all about. It’s the first step in getting your website noticed by search engines.

Indexing is the process where search engines store and organise the information they’ve gathered during crawling. Once a page is indexed, it becomes eligible to appear in search results. If your pages aren’t indexed, they won’t show up when people search, no matter how great your content is.

How Can I Make Sure My Site is Optimised for Crawling?

Create and maintain a clean site hierarchy

Site architecture, or how your pages are linked together, plays a huge role in how easily search engines can navigate your site. A well-organised structure helps crawlers quickly find and understand your content. Keep it simple: make sure every page is just a few clicks away from your homepage. Like this:

In the example structure above, every page is arranged in a clear hierarchy. The homepage connects to category pages, which then link to individual subpages. This setup not only keeps things organised but also helps avoid orphan pages (stray pages with no internal links). Orphan pages are tough for both crawlers and users to discover, so it’s best to structure your site to keep everything connected.

Submit your Sitemap to Google

A sitemap is a file that lists all the important pages on your website, acting like a roadmap for search engine crawlers. It helps them discover and prioritise your content, ensuring nothing important gets overlooked. Sitemaps are especially useful for larger websites or those with complex structures.

It’s often found on either of these URLs:

• yoursite.com/sitemap.xml
• yoursite.com/sitemap_index.xml

Using Google Search Console (GSC), you can submit your sitemap to Google, signalling that you want it to start indexing your site. This makes it easier for search engines to find and crawl your pages, helping your content show up in search results faster.

Follow these simple steps to submit your sitemap:

• Step 1: Log into GSC to get started. If you haven’t already, you’ll need to verify that you own the site you’re submitting. Not sure how to do this? There’s a helpful guide on SEMrush that walks you through the process step by step.
• Step 2: Once validated, in GSC, click “Indexing” > “Sitemaps” from the left hand side sidebar
• Step 3: Input your sitemap URL and then click submit
• Step 4: That’s it! A success message means Google has processed your sitemap and is ready to crawl your site.

Fix Broken Links

Broken links are links on your website that lead to non-existent pages, resulting in a “404 error.” They can occur when a page is deleted, the URL changes without a proper redirect, or there’s a typo in the link itself.

They can frustrate users and disrupt the user experience, often causing visitors to leave your site. Search engines also frown upon them, as they signal poor site maintenance and can negatively impact your rankings as they interrupt the crawling process.

Thankfully, tools such as SEMrush, Moz, or Ahrefs make it easier to find broken links on your site. Google Search Console can also flag them under the “Coverage” section.

Upon auditing, here’s how you can fix broken links:

• Internal links: Update the URL to the correct destination or create a 301 redirect if the original page no longer exists.
• External links: Replace the link with a valid one or remove it altogether if no alternative exists.
• Regular maintenance: Periodically check for broken links using your preferred SEO tools to keep your site clean and user-friendly.

Fixing broken links helps both users and crawlers navigate your site seamlessly, improving your overall SEO performance.

Improve Site Speed

Page speed is a crucial factor in technical SEO. Fast-loading pages provide a better user experience, reduce bounce rates, and keep visitors engaged. Search engines, like Google, also use page speed as a confirmed ranking factor, meaning slow pages can harm your position in search results.

Users expect pages to load instantly, if they don’t, they’re likely to leave. This not only impacts your conversions but also signals to search engines that your site isn’t providing a positive user experience, leading to lower rankings and less traffic. In fact, a study from Unbounce found that 70% of consumers say loading time influences their decision to make a purchase.

There are plenty of tools available to help you understand your site’s performance and identify ways to improve it. Here are some of the most popular:

• Google PageSpeed Insights: Provides performance scores along with actionable recommendations to boost your site speed.
• GTmetrix: Delivers detailed reports on load times and highlights which elements are slowing down your site.
• DebugBear: Analyses site performance over time and offers insights into how changes affect speed and user experience.
• WebPageTest: Tests your site’s speed from various locations and devices, giving you a well-rounded view of performance.

PageSpeed Insights

Of the tools mentioned above, PageSpeed Insights (PSI) is the most widely used. While it’s a great resource for identifying areas of improvement, it doesn’t always provide a full picture of real-world user experiences.

Firstly, PSI collects data from those users who:

• Use Chrome on desktop or Android (no data is collected on iOS)
• Are logged into their Google account
• Have opted into usage statistics reporting

This does not cover all user experiences and in addition to this, it’s also unclear how data from users is aggregated across the metrics (whether a small subset of users who are very active are skewing the overall metrics). According to DebugBear:

“The primary number reported by Google is the 75th percentile of experience. So if your Largest Contentful Paint (LCP) is reported as 3 seconds then that means that 75% of users had an LCP below 3 seconds and 25% had an LCP that took more than 3 seconds.”

Secondly, the Google lighthouse repository suggests that PSI tests are run using the following throttling:

• Latency: 150ms
• Throughput: 1.6Mbps down / 750 Kbps up.

The speeds referenced above roughly align with the slowest 25% of 4G connections and the fastest 25% of 3G connections. This highlights a disconnect between PSI results and the majority of user experiences, as 4G and 5G are now the standard. In summary, while PSI offers valuable insights, its results should be viewed with these limitations in mind. That said, the tool does effectively highlight areas of opportunity worth addressing. To get a more accurate and comprehensive view of your site’s performance, it’s always a good idea to compare PSI data with insights from other tools, like those mentioned earlier.

Google Core Web Vitals

To really get a grip on how your website performs in the eyes of Google, it’s important to understand Google Core Web Vitals. These 5 metrics play a key part of what Google uses to assess the user experience on your site. But what exactly are they, and why do they matter?

Largest contentful paint (LCP)

LCP measures page loading speed, focusing on how quickly the primary content of a webpage becomes visible to users. Google recommends maintaining an LCP under 2.5 seconds, which directly impacts user perception of site performance and search ranking potential, learn more here.

Interaction to Next Paint (INP)

INP is a metric that assesses a page’s overall responsiveness to user interactions by observing the latency of all click, tap, and keyboard interactions that occur throughout the lifespan of a user’s visit to a page. The final INP value is the longest interaction observed, learn more here.

Cumulative layout shift (CLS)

CLS tracks visual stability by monitoring unexpected page movements during loading. A low CLS score below 0.1 prevents frustrating user experiences where page elements suddenly jump around, potentially causing accidental clicks or disorientation, learn more here.

First Contentful Paint (FCP)

FCP is a key user-centric metric that measures how quickly users see the first piece of content on a page. It marks the point in the loading process when users can visually confirm that the page is loading. FCP accounts for 10% of a website’s overall performance score. A quick FCP reassures users that the page is on its way to loading, improving their overall experience, learn more here.

Total blocking time (TBT)

TBT tracks the total amount of time that a page is blocked from responding to user input, such as clicks or typing. TBT measures how long it takes for JavaScript execution or other processes to finish and allow user interaction. Keeping TBT under 300 milliseconds helps ensure smooth user interaction with your site, learn more here.

These Core Web Vitals, when optimised, can significantly enhance the user experience on your website and improve your search rankings in Google, making it essential to keep them in mind during your optimisation efforts.

Right, So How Can I Speed Up My Site?

Generally, there are two main types of optimisation strategies: non-technical and technical.

Non-technical Strategies

• Compress images and use modern formats: Tools like CloudConvert and Compressor.io can help reduce image file sizes without compromising quality. Always aim for modern formats like WebP and AVIF, as they offer faster loading times and better compression.
• Avoid iframes: Iframes can slow down page load times because they require additional resources to load external content. Instead, embed content directly or use alternative methods like JavaScript embeds instead.
• Limit web fonts: Too many custom fonts can increase page load times. Stick to system fonts where possible, or limit the number of custom fonts and weights you use.
• Reduce redirects: Each redirect creates an additional HTTP request-response cycle, slowing down your site. Minimise redirects and ensure that links point directly to the final destination.
• Prioritise above-the-fold content: Make sure that visible content (above the fold) loads first to improve user experience. This ensures users can start engaging with your site while the rest of the content continues loading in the background.

Technical Strategies:

• Minify any JavaScript or CSS file: Reduce the size of CSS, JavaScript, and HTML files by removing unnecessary characters.
• Enable caching: Use browser caching to store static resources, so they don’t need to be loaded every time a user visits.
• Use a content delivery network (CDN): Distribute your content across multiple servers worldwide to improve load times, especially for global visitors.
• Upgrade hosting: If your current hosting provider is slow, consider upgrading to a faster plan or a better provider to reduce server response time.
• Defer non-essential JavaScript: Load JavaScript files only when necessary to avoid blocking the page’s render time.
• Implement Lazy Loading: Load images, videos, and other resources only when they enter the user’s viewport (visible area), reducing initial load times and saving bandwidth.

How Can I Make Sure My Site is Optimised for Indexing?

Be Cautious When Using the Noindex Tag

The “noindex” tag is an HTML element that prevents your pages from being included in Google’s index.

It’s added to the <head> section of your webpage and appears like this:

<meta name=”robots” content=”noindex”>

You generally want all your key pages to be indexed, so reserve the use of the noindex tag for when you need to exclude specific pages. Examples include:

• Thank you pages or pages that visitors are redirected to after form completion
• Cart or checkout pages
• Internal search result pages
• Login or account pages (where content isn’t publicly accessible)

You can also add the nofollow and noarchive attributes to gain more control over how bots interact with your pages:

• Nofollow
  • <meta name=”robots” content=”nofollow”>
  • Tells search engines not to follow the links on the page, preventing link juice from being passed to other pages.
• Noarchive
  • <meta name=”robots” content=”noarchive”>
  • Prevents search engines from showing a cached version of the page in search results. Useful for time-sensitive or internal documents.

Prevent Duplicate Content Issues with Canonical Tags

A canonical tag is an HTML element that helps you tell search engines which version of a page is the “preferred” or “original” version, especially when there are multiple pages with similar or duplicate content. It helps prevent issues with duplicate content, which could otherwise harm your site’s SEO.

The canonical tag is placed in the <head> section of a webpage and looks like this:

<link rel=”canonical” href=”https://www.example.com/preferred-page”>

This tag informs search engines that the URL specified in the href attribute is the preferred version of the page, consolidating any ranking signals to the canonical page and avoiding potential penalties for duplicate content.

Make sure your site is using HTTPS

HTTPS (Hypertext Transfer Protocol Secure) is the secure version of HTTP, designed to protect sensitive information like passwords and credit card details from being intercepted.

Since 2014, HTTPS has also been a ranking factor for search engines.

To see if your site uses HTTPS, just visit it and look for the “lock” icon in the address bar, it’s your sign that the connection is secure.

Technical SEO Checklist

Looking for a straightforward guide? See our comprehensive technical SEO checklist, it’s designed to help you tackle potential technical issues and ensure your users have the best possible experience.

Crawlability and Indexability

1. Redirect or replace broken internal links
2. Fix 404 and 500 errors
3. Fix redirect chains and loops
4. Use an XML sitemap
5. Set up your robots.txt file
6. Make sure important pages are indexed

Website Structure

1. Check your site structure is organised
2. Optimise your URL structure
3. Add breadcrumbs
4. Minimise your click depth
5. Identify orphan pages

Accessibility and Usability

1. Make sure you’re using HTTPS
2. Use structured data
3. Use hreflang for multi-lang pages

Speed and Performance

1. Improve your Core Web Vitals
2. Ensure mobile-friendliness
3. Reduce the size of your webpages
4. Optimise your images
5. Remove unnecessary third-party scripts

Content

1. Address duplicate content issues
2. Fix thin content issues
3. Check your pages have metadata

Key Takeaways

Technical SEO forms the backbone of a well-optimised website, making sure it’s not only visible to search engines but also offers an exceptional experience to users. Focusing on aspects like site speed, mobile usability, and crawlability can significantly impact both your rankings and your visitors’ satisfaction.

Tools like Google PageSpeed Insights (PSI) and metrics such as Core Web Vitals provide a solid starting point to measure performance and identify areas for improvement. However, technical SEO isn’t a one-and-done task, it requires regular audits and ongoing maintenance to address new challenges and stay ahead of evolving search engine algorithms.

By staying proactive and leveraging the right strategies, you can create a site that consistently delivers value, both to users and search engines, ensuring long-term growth and success.