What SEO Audits You Can Do in Netpeak Spider

Use Cases
2Like
1Comments
Share
What SEO Audits You Can Do in Netpeak Spider

I realize that sometimes Netpeak Spider settings that help deal with a specific range of tasks more effectively may stay out of sights. That’s why I made this blog post to help scratch beneath the surface and show different types of website audits. It‘ll give you insights into how you can solve specific tasks and get various reports for a single project.

  • 1. Quick Audit ‘with Search Robot’s Eyes’
  • 2. Audit for the Crawl Budget Optimization
  • 3. Audit at the Start of Website Optimization
  • 4. Audit at the Pre-Sale Stage
  • 5. Audit of the External Links
  • 6. Audit for Websites That Moved to HTTPS Protocol
  • 7. Audit of Website Speed
  • 8. Audit of Website Load
  • 9. Audit of Metadata
  • 10. Audit of Images and Media Files
  • 11. Audit of Giant Websites
  • 12. Audit of Multilingual Website
  • 13. Audit of the Group of Websites (Multi-Domain Crawling)
  • 14. Internal Linking Audit
  • 15. Audit of Optimization Priority
  • Recap

Netpeak Spider crawler has a free version that is not limited by the term of use and the number of analyzed URLs. Other basic features are also available in the Freemium version of the program.

To get access to free Netpeak Spider, you just need to sign up, download and launch the program 😉

Sign Up and Download Freemium Version

P.S. Right after signup, you'll also have the opportunity to try all paid functionality and then compare all our plans and pick the one most suitable for you.

1. Quick Audit ‘with Search Robot’s Eyes’

In this audit, you can assemble the pages that get under the search robot’s ‘view’ in the first place. So you'll know what issues on certain pages need to be fixed sooner than others. I’ll showcase each type of audit.

First, enable the settings that help the program imitate a search robot’s behavior:

  1. In the 'Advanced' settings, select the 'Default: bot' template, which triggers all crawling and indexing instructions needed for this audit.

    How to enable all crawling and indexing instructions in Netpeak Spider

  2. If the target website uses JavaScript to display part of the content, or it’s an SPA website, turn on JavaScript rendering on the ‘General’ settings tab.

    How to enable JavaScript rendering in Netpeak Spider crawler

  3. On the 'User Agent' tab choose a search engine robot which is a focus for your website promotion: 'Yandex Bot', 'Bingbot', 'Googlebot', etc.

    How to choose a search bot in Netpeak Spider

  4. Search robots don’t crawl the entire website during one session, especially if the website is large. That’s why we’ll set restrictions for the maximum URL depth and maximum crawling depth.

    How to set restrictions on crawling depth in Netpeak Spider

When the settings you need are on, start crawling. In the end, you'll see the main table only with those pages which are likely to be included in the search engine’s index.

Received data can be enriched with pages from a sitemap. Use a built-in instrument ‘XML sitemap validator’ for that purpose. Also, you have the opportunity to improve the report with data from Google Analytics and Search Console that shows traffic, clicks, impressions, which will help define:

  • compliant pages without traffic, clicks, or impressions
  • non-compliant pages with traffic, clicks, and impressions

More about integration with Google Analytics and Search Console read in the User Guide.

Netpeak Spider doesn’t let the same URLs get to the main table. If GA, GSC, or sitemap contain pages similar to those in the table, they won’t slip in the table. Only pages that weren’t detected during crawling will be added. Kept in sight these reasons why the pages can still be missed out:

  • The search robot doesn't see links to these pages. For instance, to open the link, you need to click on a particular button.
  • There are no links to these pages on the website.
  • The links are nested too far from the homepage. In this case, you'll need to optimize the crawl budget. We’ll approach this type of audit in the next example.

Download PDF Report

2. Audit for the Crawl Budget Optimization

In this type of audit, we’ll use segmentation. Before crawling, opt for any settings you need, but remember to select these parameters:

  • ‘Status Code’
  • ‘Crawling and Indexing’ checkbox
  • ‘Links’ checkbox
  • ‘Click Depth’

After the crawling is complete, check out these issues:

  • Compliant pages with low PageRank. Compliant pages should get more link equity than non-compliant pages because they potentially can bring traffic to the website. Consequently, you should detect pages that receive an insufficient amount of link equity and place more internal links that will lead to them. To detect such pages, apply a segment and filter PageRank value in the ascending order, as shown on the screenshot.

    How to find compliant pages with low PageRank in Netpeak Spider

  • Non-compliant pages with high PageRank.
  • Pages that dwell on the click depth far from the '5' value from the initial page. To spot what important pages lie far from the initial one, use a click depth segment.

    How to find pages that are far from the main page using the Netpeak Spider

  • Pages that don’t receive link equity.

Also, I recommend bearing in mind the links that lead to the pages with redirects. They also deplete the crawl budget.

In the 'Export' menu, you can export a detailed report on redirects.

How to export a detailed report on redirects from Netpeak Spider

Download Report

As in the previous audit, you can extend data with links from the sitemap to deal with the ones that weren't found during crawling.

3. Audit at the Start of Website Optimization

This type of technical audit is the opposite of the 'with search bot's eyes' audit because when you begin optimizing a project, it's essential to give it a heavy kick in the rankings at the very start. To that end, you should find the maximum optimization issues and fix them sooner better than later.

To do so:

  • Enable the crawling of the external links.

    How to enable crawling of external links in Netpeak Spider

  • Disable crawling and indexing instructions.

    How to disable crawling and indexing instructions in Netpeak Spider

  • Reset restrictions and rules (if they’re set)
  • Enable all parameters.

    How to enable all parameters in Netpeak Spider

With these settings in action, the program will check website pages for all available SEO issues. To export all reports on the detected issues, use the 'Export' menu.

How to export all reports on issues in Netpeak Spider

For quality SEO audit at the starting line, it’s important to narrow down on fixing:

  • broken links
  • incorrect content in the canonical tag
  • duplicate content (title, description, <body> tag)
  • empty title and description
  • long server response time
  • redirects with bad URL format

To find reports on these issues, go to the 'Special issue report' in the 'Export' menu. Speaking of the 'Special issue report,' you can find an 'Issue overview + descriptions' report, which can be used as a ground for a technical task for developers. If needed, export this report.

How to export 'Issue overview + descriptions' report from Netpeak Spider

Download Report

4. Audit at the Pre-Sale Stage

If you have a few hot minutes to demonstrate the client what major optimization shortfalls drag down the website performance and give recommendations on tackling them, you’ll need:

  • White label report
  • The issue overview list with issues detected during crawling, their descriptions, and suggestions on how to handle them.

To get this report, you should go to the ‘Export’ menu.

How to export 'Special issues report' in Netpeak Spider

To sound persuasive at the pre-sale stage, carry out the audit with the settings from the previous audit example so that the program crawls all pages and assembles the maximum of all possible errors.

Download Report

Internal linking optimization is crucial, but don’t forget about optimization of external links. This audit will help you track down low-quality external links, namely:

  • broken links
  • links that lead to the pages with long server response time
  • links to low-quality websites and websites with malicious resources (the checks are conducted in Netpeak Checker).

To do this audit in Netpeak Spider:

  1. Enable crawling of the external links

    How to enable crawling of the external links in Netpeak Spider

  2. Tick the parameters: 'Status Code' and 'Response Time'.

When the crawling is completed, apply the segment for the external links.

How to set a segment by external links in Netpeak Spider

In the end, you'll get the list of all external links and reports on issues that were found on the pages your website refers to.

To swing into high gear, you can open this report in Netpeak Checker to ensure that none of these pages is on Google's 'blacklist'. Google Safe Browsing server will help you find that out.

How to check links using Google Safe Browsing service in Netpeak Checker


Download Netpeak Spider and Checker Reports

6. Audit for Websites That Moved to HTTPS Protocol

After your website moved from HTTP to HTTPS protocol, it’s important to ensure that there are no HTTP links left, direct links to HTTPS are set without redirects, and the mixed content issue doesn’t jeopardise your website security.

Before crawling, enable these parameters:

  • ‘Status Code’
  • ‘Incoming Links’
  • ‘Redirects’
  • ‘Target URL Redirect’
  • ‘Page Hash’
  • Outgoing Links’ checkbox to fish out the mixed content

We described a detailed algorithm for detecting and fixing mixed content issues in this blog post: 'How to Check Website for Mixed Content with Netpeak Spider.’

In the settings, you can also enable the crawling of the additional pages and files (JS, CSS, etc.).

How to enable crawling of service pages and files in Netpeak Spider

When the crawling is completed, you look for report on the links with HTTP protocol on the ‘Overview’ tab.

How to open a report on links with the HTTP protocol in Netpeak Spider

To see the list of incoming links for one page, right-click to summon the context menu or use the F1 hotkey. To view all incoming links for all pages in the report → Shift+F1.

You can also check out the report, which shows the URLs with redirects, where they lead, and the incoming links on these pages.

How to view a report on URLs with redirects in Netpeak Spider

You’ll find it this way in the technical SEO audit (PDF):

  • The total number of links that lead to the pages with redirects and the number of broken links.

    The total number of links to pages with redirects and the number of broken links in the SEO audit (PDF) in Netpeak Spider

  • The number of pages with HTTP protocol.

    The number of pages with the HTTP protocol in the SEO audit generated in Netpeak Spider

Download Report

7. Audit of Website Speed

Website speed is an indispensable factor in search engine optimization and one of the most significant ranking factors. Especially in the wake of Core Web Vitals coming into play.

To analyze website speed, crawl your website with these parameters turned on:

  • ‘Status Code’
  • ‘Response Time’

    The parameters that need to be enabled for speed audit in Netpeak Spider

In such a way, you'll crawl the website in a blip of an eye with your device's minimum resource consumption. As a result, you'll get information on how fast the server responds to requests and what issues may occur with the server response time.

Reports on the server response speed in Netpeak Spider

By default, the pages that respond more than 500 mc in Netpeak Spider fall into the report on the 'Long Server Response Time' issue. However, you can set the limits on the 'Settings' tab → 'Restrictions'.

How to set restrictions in Netpeak Spider

Additionally, you can check these pages in the 'Google Pagespeed Insights' tool. To do so, right-click on the target URL and choose the service.

How to check URLs in the 'Google Pagespeed Insights' tool straight from Netpeak Spider

Also, you have the opportunity to conduct a bulk URL check in the PageSpeed Insights service using Netpeak Checker. Transfer URLs from Netpeak Spider to Checker, select desired parameters in a sidebar, and start the analysis.

How to perform bulk URL check in the PageSpeed Insights service using Netpeak Checker

In technical SEO audit, you'll see this information about server response time.

Information about server response time in technical SEO audit generated in Netpeak Spider

Server response time and content loading time can vary according to the load on the website server. For instance, the pages load at flying speed if there are few visitors on the website at the moment, but if there are more visitors, the server capacity to load fast slows down.

In Netpeak Spider, you can increase the number of requests which are sent to the website server, thus creating a denser load on the server and analyzing how fast the requests are processed. We’ll peruse this case in the next audit type.

Download Report

8. Audit of Website Load

Before you start an audit, turn off all parameters except for those that you need to check website speed. Crawl website with the maximum number of threads. It will help you understand how the server behaves under the high load strain.

Apart from the information about speed, I recommend paying your attention to the status codes and check if any pages with ‘Timeout’ status code were spotted.

9. Audit of Metadata

To check the accuracy of metadata (website content audit), turn on these parameters before crawling:

  • ‘Status Code’
  • ‘Meta Robots’
  • ‘Title’
  • ‘Title Length’
  • ‘Description’
  • ‘Description Length’
  • ‘H1 Content’

In the settings, enable all crawling and indexing instructions.

How to enable crawling and indexing instructions in Netpeak Spider

As a result, the tool will check pages for the issues related to meta tags:

  • ‘Duplicate Title’
  • ‘Duplicate Description’
  • ‘Duplicate H1’
  • ‘Missing or Empty Title’
  • ‘Missing or Empty Description’
  • ‘Multiple Titles’
  • ‘Multiple Descriptions’
  • ‘Same Title and H1’
  • ‘Max Title Length’
  • ‘Short Description’
  • ‘Max Description Length’
Information about content optimization in the SEO audit generated in Netpeak Spider
Information about content optimization in the SEO audit generated in Netpeak Spider


Download Reports

10. Audit of Images and Media Files

Optimization of images is yet another important aspect of website promotion that you should keep a weather eye on. That's why it's important to fall in line with search engines' recommendations that concern visual content on the webpages. Netpeak Spider checks these major issues with images and other media files (audio, archives, etc.)

  • ‘Images without ALT Attributes’
  • ‘Max Image Size’
  • ‘Other SEO Issues’

For media files, general issues are detected (redirects, broken links, etc.)

To do an audit of the issues related to images and audio files, it's enough to crawl the website with default program settings.

After the crawling is completed, you can filter results with the help of the segmentation feature. It's shown on the example screenshot below.

How to filter results using the segmentation feature in Netpeak Spider


Export Report

11. Audit of Giant Websites

If you fear that your computer data storage resources are limited, follow the recommendations below.

The more parameters are engaged during crawling, the more RAM is spent on displaying results. To decrease the likelihood of an error due to the shortage of RAM, crawl the website twice.

During first crawling, disable all parameters, except status code. Thus the program will quickly find all website pages without a significant load on RAM. When the crawling is completed, save the project.

Before you start crawling for the second time, clear the address bar, go to the 'Parameters' tab, tick the necessary points, and hit 'Restart'.

How to recrawl website in Netpeak Spider

In this case, the crawling will be carried out according to the URLs list, which is less resource-consuming than crawling in the standard mode.

As far as the crawling continues, you'll see the data on the selected parameters appearing in the results table.

12. Audit of Multilingual Website

If the website targets the audience from different countries, multilingual content is a must. The website may have several alternative language versions for each page. For search engines not to take them for duplicates, it's vital to specify website versions in foreign languages in the hreflang attribute. The use of this tag entails a range of rules, that's why it's easy as ABC to make a mistake.

In Netpeak Spider, you can figure out whether or not all conditions have been met. In case the program notices that you failed, it will point to the issue source in the corresponding reports.

Before you start the audit, it’s enough to enable two parameters from the hreflang group: ‘Hreflang Language Code’, ‘Hreflang Links’ and also the ‘Status Code’ parameter.

The crawler will check this tag, and the spotted issues will be displayed in corresponding reports.

How to check hreflang tag in Netpeak Spider

As a result, you’ll check if:

  • the current pages contain hreflang links
  • the hreflang attribute doesn't include broken links
  • the language codes are correct and there’s no duplicate codes
  • there’re alternative URLs without duplicates
  • there’re confirmation links in the hreflang attribute
  • hreflang is clear from relative links and / or links to non-compliant URLs
  • language code in confirmation hreflang links is consistent

Besides, you can find a separate report on the links from the hreflang attribute in the program. You can open it in the 'Database' menu.

How to open a separate report on links from the hreflang tag in Netpeak Spider

If a separate domain is used for each language, audit the group of websites to crawl all website versions.

Download Report

13. Audit of the Group of Websites (Multi-Domain Crawling)

To crawl several domains in different languages:

  1. Make sure that the hreflang attribute check is enabled.
  2. Insert the URL of one of the website domains into the main table and crawl it.
  3. Open the hreflang report, copy the list of all domains and transfer them to the main table.
  4. Go to the ‘General’ settings, enable the multi-domain crawling and start the program.

    How to enable multi-domain crawling in Netpeak Spider

In this mode, the program will crawl all pages from domains mentioned in the main table during one session. Thus you'll gather full-package information about all language domains of your website.

Note that the program will mix the pages from different domains in the issue reports. If you want to export reports on each domain separately, use segmentation.

For instance, to get reports separately, you should go to the 'Overview' tab, choose a necessary domain, and apply it as a segment.

How to apply a domain as a segment in Netpeak Spider

Now you can approach all parameters in the table and export the examples of audit reports for a picked domain. The same steps should be repeated for the remaining domains.

Multi-domain crawling is a Pro feature of Netpeak Spider. Eager to have access to this and other PROfessional features? They include:

  • white label reports that allow branding
  • export of search queries from Google Search Console and Yandex.Metrica
  • integration with Google Drive / Sheets, etc.

Hit the button to purchase the Pro plan, and get your inspiring insights!

Buy Netpeak Spider Pro and Feel Its Power

14. Internal Linking Audit

Internal linking does its bit into website usability and distributes link equity so that search engines could index landing pages better.

To carry out this audit, it's enough to choose the parameters template 'For PageRank'.

How to check internal linking in Netpeak Spider

The program will crawl the entire website and fetch all necessary information about internal linking, including:

  • Internal links and external links with anchor texts, values in the rel attribute and type of the link in the source code of the page. The reports are stored in the 'Database' module.

    Internal linking information collected by Netpeak Spider

  • Information about the distribution of link equity inside the website: the number of incoming links that each page gets, important pages that receive insufficient link equity, and vice versa, unimportant pages with high PageRank.

Also, the 'Internal Pagerank calculation' tool will show the 'dead ends' which burn down the link equity and how it's distributed across the website in general.

More details about this instrument: 'Internal PageRank from A to Z.'

15. Audit of Optimization Priority

The audit of the optimization priority will help detect pages that receive an insufficient volume of organic traffic compared to paid traffic. To find such pages, you should complete two tasks:

  • Task #1. Find pages which receive tons of paid traffic compared to organic.
  • Task #2. Among these pages, find those ranked poorly with Google Search Console and analyze parameters in Serpstat and Netpeak Checker.

Let’s get our hands a bit dirty 😃

Task #1

  1. Add Google account in Netpeak Spider. We've described how to add your Google account in the article: ‘Integration with Google Analytics и Search Console.'
  2. Crawl the website or upload the URLs from Google Analytics (look at the 'List of URLs menu')

    How to upload pages from Google Analytics into Netpeak Spider

  3. Set paid traffic in the segment dropdown menu in the Google Analytics settings.

    How to set paid traffic in Google Analytics settings in Netpeak Spider

  4. Export data on sessions into the main table via the 'Analysis' menu.

    How to export data on sessions from Google Analytics in Netpeak Spider

  5. Export the received report.
  6. Set the organic traffic in the segment settings and repeat the steps described above.
  7. Merge two reports and compare the balance of paid and organic traffic. In the end, you’ll figure out what pages get more paid traffic over organic.

    How to compare the balance of paid and organic traffic in Netpeak Spider

Task #2

Upload the crawled URLs into the new project in Netpeak Spider and enable the ‘GSC: Average Position’ parameter.

How to upload the crawled URLs into the new project in Netpeak Spider

If you have an active account in Serpstat, you can upload the URLs list and get information about the number of words the website ranks for on particular positions in Google. To do so, turn on the parameters depicted on the screenshot.

The parameters you need to set in Netpeak Checker to get information about the number of words the website ranks for on particular positions in Google

Result:

The words the website ranks for on particular positions in Google

Recap

By combining different crawling settings and parameters, you can flexibly tailor Nepteak Spider, automatically detect specific optimization issues and get reports relevant to your tasks. This will help you conduct a complex website audit and tackle detected issues more quickly and effectively in the future.