65+ Ways To Use WebSite Auditor

By default, WebSite Auditor will crawl the pages and resources of your site that are allowed for all bots (the * user agent) in your robots.txt. If you'd like to crawl and audit all pages of your site, regardless of robots instructions, you can easily set that up in the tool. All you need to do is create a project (or, if you are doing this for an existing project, click on Rebuild Project) and check the Enable expert options box at Step 1.

At Step 2, uncheck the Follow robots.txt instructions box and hit Next.

In a few moments, all of the pages and resources on your site will be collected and you'll find yourself in the Site Audit dashboard, which highlights the errors and warnings, if any, found on your site.

WebSite Auditor lets you crawl websites following the robots.txt instructions for a specific bot of your choice. To do this, you'll need to create a project (or use the Rebuild Project button to rebuild an existing one) and make sure the Enable expert options box at Step 1 is checked. At Step 2, make sure the Follow robots.txt instructions option is enabled, and click on the drop-down menu next to it. Select the bot you'd like to crawl your site as (if the bot you're looking for isn't on the list, feel free to specify a custom one). Finally, hit Next to proceed with the crawling.

As you are auditing a site, you might sometimes want to use a specific user agent to crawl your site as. This can be useful if you'd like to audit the mobile version of a responsively designed site just as Google's mobile bot sees it, or if you're looking to identify instances of cloaking on a site.

To do this, create a new project, making sure the Enable expert options box is checked at Step 1. At Step 2, check the Crawl as a specific user agent box. In the drop-down menu to the right, pick a user agent to use. Remember to also keep the search engine bot whose user agent you're about to use specified under the Follow robots.txt instructions option.

If you're looking to choose Google's user agent for smartphones for example, pick the following user agent:

Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

For a full reference on Google crawlers and user agents, jump here. For a list of Bing's user agents and bots, click here.

If you need to, you can crawl some directories (or pages) on your site instead of the entire website.

If you're looking to crawl only one specific directory on your site, all you need to do is create a new project and enter the URL of the folder without tweaking any additional settings. WebSite Auditor will crawl only pages within that folder.

If you'd like to crawl several directories or pages with certain symbols in the URL, you'll need to create or rebuild a project (again, make sure the Enable expert options box is checked at Step 1). At Step 2, go to the Filtering tab and check the first box. In the field right below, specify the folders you're looking to crawl, separated by space.

Sometimes you'd want to only crawl a part of your site and exclude some pages or directories from the crawl (say, if you're looking for a quick audit of your most important pages and don't want your dashboard cluttered with all your forum pages).

To do this, you'll need to create or rebuild a project, and check the Enable expert options box at Step 1 of project creation/rebuild. At the next step, switch to the Filtering tab and check the Do not scan pages… box (if you need to, you can use this option in combination with the Collect only pages… filter above; otherwise, you'd want to leave the first checkbox disabled). In the field below, specify the folders/pages you'd like to exclude from your crawl, separated by space.

If your website includes subdomains, you can crawl them along with your main domain in WebSite Auditor. This will allow you to audit and analyze all your pages and resources — and your site structure as a whole — in a single project.

To crawl a domain and its subdomains altogether, simply create a new project or rebuild an existing one, and click to Enable expert options at Step 1. At the next step, go to Advanced Options tab and tick the Crawl Subdomains box.

If you wish to crawl a single subdomain on your site, simply create a new project and enter the URL of your subdomain, without specifying any additional settings.

WebSite Auditor can now execute JavaScript and crawl dynamic content — just the way modern browsers (and Googlebot) do. That means that the tool is now able to render and fully crawl AJAX-based sites and any content generated with JavaScript (including copy, links, and entire pages).

To enable rendered crawling, you'll need to create or rebuild a project, and check the Enable expert options box at Step 1. At Step 2, you should go to the Advanced Options tab and check the Execute JavaScript option.

Pages with various parameters appended to the URL can pose sometimes a problem, especially with sites powered by a CMS that might be generating the parameters automatically, thus confusing search engines and potentially causing content duplication concerns. WebSite Auditor lets you crawl your site and treat the same page with different parameters as different pages so you can spot any unnecessary parameters in your URL and get rid of them to keep your URLs clean.

Create (or rebuild) a project in WebSite Auditor, with the Enable expert options box checked at Step 1. At Step 2, go to the URL Parameters tab and uncheck the Ignore URL parameters box. Click Next to proceed with the scan.

If you're looking to get a list of pages with certain parameters in the URL (say, Google Analytics parameters or session IDs), you'll need to set up your crawl accordingly. Create (or rebuild) a project in WebSite Auditor, with the Enable expert options box checked at Step 1. At Step 2, switch to the Filtering tab and specify the parameter you're looking for (if you're entering several parameters, make sure to separate them by space).

Now, go to the URL Parameters tab and uncheck the Ignore URL parameters box. Click Next to proceed with the scan.

When your crawl is complete, navigate to the Pages dashboard and enter the parameter you're looking for in the quick search box. In the filtered view, you'll see all URLs on your site that contain the parameter you specified.

Old servers (and some of the cheaper hosting plans) may not be able to handle the number of concurrent requests sent by spiders. To deal with this, you can change your crawl speed in WebSite Auditor. Create or rebuild a project, and make sure the Enable expert options box is ticked at Step 1. At Step 2, go to the Speed tab, check the Limit the number of requests to the website box, and specify the number of requests per second to be sent to your site.

With WebSite Auditor you can crawl sites, certain subdomains or directories that require authentication, and optimize password-protected pages. You should simply create a new project or rebuild yours, and click to Enable expert options at Step 1. At Step 2, go to Advanced Options tab, tick the Crawl password-protected sites box and specify your login details.

By default, WebSite Auditor will collect all kinds of internal and external resources used on your site, not just its HTML. Once you've created your project, go to the All Resources dashboard under Site Structure and click on JavaScript, CSS, Images, or any other kind of resource you're about to look into. Here, you'll see a full list of resources of every type, along with each resource's HTTP response code, robots instructions, and the number of pages it was found on.

You can view a list of pages with 4xx/5xx status codes in the Site Audit dashboard of your WebSite Auditor project, under the Indexing and crawlability factors. If any such resources are found, you'll see a list of them on the right. Check with the Found on pages column to see which pages link to these resources.

In the Site Audit submodule, look for Pages with 302 redirect and Pages with 301 redirect under the Redirects section. If any such pages are found, you'll get a list of them on the right, along with the URL they redirect to, and the number of internal links pointing to them.

Whenever you create a new project in WebSite Auditor (or rebuild an existing one), you can check if there are any long redirect chains on your site. You should simply go to Site Structure > Site Audit and scroll down to Pages with long redirect chains section.

If any redirect chains or loops are found, you'll see a list of pages the redirects are found on, along with the full redirect paths — so you can quickly see what needs to be fixed and where.

You can view the HTTP header of any page and resource right in your WebSite Auditor project. Just go to the All Resources dashboard, select a resource, and switch to Response Header in the lower part of your screen.

To make sure HTTP/HTTPS and www/non-www versions of your site are set up correctly, take a look at those factors in the Site Audit submodule, under the Redirects section. If any problems are found, you'll get detailed how-to-fix advice in the right-hand part of your screen.

To see which pages of your site have a canonical URL set up, and what that URL is, click on Pages with rel="canonical" under the Redirects section of your Site Audit dashboard. On the right, you'll also see the pages' titles and canonical URLs.

Instances of more than one canonical URL on a single page can happen with some content management systems, when the CMS automatically adds a canonical tag to site's pages. Multiple canonical URLs will confuse search engines and likely cause them to ignore the canonical elements altogether. Check if there are instances of this on your site under the Encoding and technical factors section of your site audit, by clicking on Pages with multiple canonical URLs.

Meta refresh is a common spam technique and not the most SEO-friendly way of redirection. In WebSite Auditor, you can see if any pages on the site you're auditing use meta refresh under the Redirects section of your Site Audit dashboard.

For a full list of dofollow external links on a site, go to the Site Audit submodule and click on Dofollow external links under the Links section. Here, you'll see the pages on which dofollow links are found, the linked-to URL, and the link's anchor text.

To get a list of all broken links on the website you're auditing, click on Broken links under the Links section (in the Site Audit submodule). Here, you'll see the pages where broken links are found (if any), the URL of the broken link, and its anchor text.

To get a full list of outgoing links from a page, switch to the Pages dashboard in WebSite Auditor and click on one of the pages in the table. Below, click on Links from page to see all links on the page. In the drop-down menu on the right, you can select to view only internal, only external, or all links.

For a list of broken links from specific pages, go to the Pages dashboard and switch to the Links & technical factors tab. Scroll a little to the right to see the number of broken links on every page in the Broken Links column. To get a full list of broken links from a certain page, click on the page in the table. Below, click on Links from page. Check with the HTTP Status Code column to identify broken links by 4xx/5xx status codes.

For detailed info on any link, switch to the Pages submodule in WebSite Auditor and click on one of the pages in the table. Below, click on Links from page to see every link on the page along with its HTTP response code, anchor text, and robots directives (nofollow/dofollow).

To find pages that aren't linked to from other pages of your site, but do exist, you'll need to rebuild your WebSite Auditor project (or create a new one). At Step 1 of project creation/rebuild, check the Show expert options box. At Step 2, select Look for orphan pages, and proceed with the next steps like normal. Once the crawl is complete, you'll be able to find orphan pages in the Pages dashboard, marked with the Orphan page tag.

To easily spot pages that are buried too deep on your site, and therefore are hard to find for users and search engines, you can use the Click Depth metric. It shows the number of clicks it takes to get to a given page from the home page. Ideally, all pages should be accessible at no more than 4 clicks, to boost crawlability and avoid losing too much PageRank.

In new projects, you can find the Click Depth column in your Pages dashboard by default. For existing projects, you`ll need to add it manually by right-clicking the header of any column.

To get a list of pages with too many outgoing links, go to the Site Audit dashboard and click on Pages with excessive number of links under the Links section. Here, you'll see the pages that have over 100 outgoing links (both internal and external).

In the Site Audit dashboard, look for Pages with 302 redirect and Pages with 301 redirect under the Redirects section. If any such pages are found, you'll also see a list of them on the right, along with the URL they redirect to and the number of internal links pointing to them. Click on the Options button next to the number of links to the page for a full list of pages that link to it. To avoid wasting too much of your crawl budget on redirected pages, you may want to change those links so that they point to the destination of the redirect immediately.

For a full list of resources used on the site (internal, external, or both), go to the All resources dashboard in WebSite Auditor and switch between All resources, Internal resources, and External resources in the list on the left for a full list of resources of each kind and details on every resource.

For a full list of resources used on any of your site's pages, switch to the Pages dashboard in WebSite Auditor and click on one of the pages in the table. Below, click on All resources to see every resource used on the page, along with its content type, HTTP response code, robots instructions, the HTML tag (or HTTP header field) this resource was found in, and more.

In WebSite Auditor's All resources dashboard, click on JavaScript, CSS, Images, Videos, Flash, PDF, or Other under Internal resources to view a full list of resources of each type and details on every resource. If your site also uses external resources of any type, click through the same listings under External resources.

To get a list of problematic images on your site, go to the Site Audit dashboard and check the Images section. Here, you'll see the images that are broken or have missing alt text, along with the URLs of pages that use those images so you can quickly fix them.

You can spot potential thin content pages in WebSite Auditor's Pages module, in the On-page tab. Scroll to the right to the Word Count column and sort your pages by their word count by clicking on the column's header. Feel free to drag and drop the Word Count column to the left to better match the content length to the URLs.

With WebSite Auditor's Custom Search option you can find all instances of any piece of content across a website — be it Google Analytics code, social sharing buttons, specific HTML tags, a script, a plugin, or a piece of text. You can also use the CSS Selector option to look for specific elements by indicating any kind of selector (using CSS syntax).

The Custom Search button can be found on the top menu bar, under Site Structure > Pages. For useful tips and ideas on using Custom Search, check the full guide here.

Using frames can pose a problem for SEO, as search engines are often unable to index content inside frames. You can get a list of pages that use Frames in WebSite Auditor's Site Audit dashboard by clicking on the Pages with Frames factor under Encoding and technical factors.

If you do have specific reasons to use frames on your site, make sure you use the NoFrames tag to provide alternate content to search engine bots. If you're concerned with how your use of frames can affect SEO, here's a detailed article on search engines and frames.

For a list of pages with W3C errors, go to WebSite Auditor's Site Audit dashboard and click on Pages with W3C HTML errors and warnings and Pages with W3C CSS errors and warnings. You'll see the list of pages with the errors in the right-hand part of your screen.

When crawling your site, WebSite Auditor can help you identify pages that are too big and thus may negatively affect SEO and user experience. Page size is calculated comprehensively by adding the size of its HTML code to the size of every resource used on the page, including images, videos, and scripts.

You'll find a list of your site's slower-loading pages (along with each page's size) right in the Site Audit dashboard under Too big pages.

You can check if your pages have sctructured data (such as Schema markup) in place under the Pages dashboard in WebSite Auditor. Go to the Open graph & structured data markup tab, locate the Structured Data Markup column, and click on its header to sort your records by the values in this column. For pages that do use structured data, you'll also see the type of structured data being used in brackets.

Structured data markup helps search engines understand what your pages are about; for certain types of businesses, it can be a crucial SEO factor to optimize - local businesses, company websites, etc. Here's a helpful guide to structured data for SEO.

Coming in September: In the next WebSite Auditor update, you'll also be able to see a list of pages without structured data markup right in the Site Audit dashboard of your projects.

To see your pages' cache date in Google, Yahoo, Bing, or Yandex, go to the Pages dashboard and select the pages you'd like to get the data for (use Ctrl + A to select all). Click on Update Factors; in the window that pops up, hit Add more ranking factors.

Here, find the External factors folder and expand it. Locate the factors you'd like to add and click OK.

Now, hit Next to proceed with the update. You may want to deselect all other factors so that the check takes less time.

When the check is complete, back in your WebSite Auditor workspace, right-click the header of any column to enter the workspace editing mode. Here, double-click Cache date in Google, Cache date in Bing, etc. in the list of available columns on the left to add them to visible columns, and click OK.

Now, scroll to the right to locate these columns you just added and see when (if at all) your pages were last cached by the search engines.

To check if any of your URLs are too long (over 115 characters) and thus hard to read for users and search engines, go to WebSite Auditor's Site Audit dashboard and look for Too long URLs in the URLs section of site factors.

To see if any of your page titles and meta descriptions are over the recommended length, check with Too long titles and Too long meta description under the On-page section of factors in the Site Audit dashboard. These lengthy titles and descriptions will likely get truncated in your pages' snippets in search results, so it's best to rewrite them to make sure they are fully visible to searchers.

To see if any of your site's pages have duplicate titles or meta descriptions, check the respective factors under the On-page section of the Site Audit dashboard. If any titles or descriptions on your site are duplicated, you'll get a list of pages where the duplication occurs.

Duplicate titles and descriptions are confusing to search engines as they wouldn't know which of the pages should be ranked in response to a given query. Often, they'd choose to rank neither page instead, so it's in your best interest to ensure all your titles and descriptions are unique.

To get a list of your site's pages with empty title or meta description tags, click on Empty title tags and Empty meta description in the On-page section of your site audit.

If your page has an empty title tag, it's unlikely that it will rank well in search results as the page's title is a strong relevance signal for search engine bots. Meta descriptions have a weaker correlation with rankings, but it's best to create a unique description for every page — otherwise, the search engines will put up your snippet automatically from the page's content, which may not always look appealing to searchers.

Open graph markup is used for the snippets of your pages when they get shared on social media. It's a good idea to ensure that your open graph titles, descriptions, and images are specified for all pages on your site that can be shared on social media. To see the contents of the open graph tags for any of your pages, go to the Pages dashboard and switch to the Open graph & structured data markup tab. Here, check with the OG Title, OG Description, and OG Image columns.

If you don't have a robots.txt file on your site, you can quickly create one with WebSite Auditor. Go to the Pages dashboard and click on the Robots.txt button. Click Add Rule to specify a new rule. You can preview the contents of your robots.txt right below. When you're done, click Next and either save the file locally on your computer or upload it to your site via FTP.

To edit your current robots.txt file, go to the Pages dashboard and click on the Robots.txt button. In the window that pops up, you'll see the list of robots instructions in the current file. Click Add Rule to specify a new rule, or Edit to modify an existing one. When you're done, click Next and either save the file locally on your computer or upload it to your site via FTP.

For a full list of pages and resources blocked by robots instructions, go to the Site Audit dashboard and click on Resources restricted from indexing (under the Indexing and crawlability section). Here, you'll see the resources that aren't available to crawlers, their status codes and content type, and where the disallow rule was found.

You can build a sitemap for your site from scratch right in WebSite Auditor. Go to the Pages dashboard and click on the Sitemap button. You'll see an automatically generated sitemap in a pop-up window; here, you can exclude pages blocked by robots instructions from the sitemap by checking the respective box, manually uncheck boxes next to pages you don't want to include in your sitemap, and adjust each page's change frequency and priority.

You can view visits to your pages right in WebSite Auditor if you sync your Google Analytics account with the app. To do this, go to Preferences > Google Analytics Account and hit Click to enter account settings. Specify your Google Analytics login and password, and click OK. Now go to the Pages dashboard and switch to the Traffic coming to pages tab. Select the pages you'd like to check visits for (use Ctrl + A to select all) and click Update Factors. In the window that pops up, click on Add more ranking factors.

Here, find the External factors folder and expand it. Select Page Visits and Page Bounce Rate and click OK.

Now, simply hit Next to proceed with the update. You may want to deselect all other factors so that the check takes less time. When it's done, you'll see the number of visits to each page in the last 30 days along with its bounce rate in the Page Visits and Page Bounce Rate columns.

To see every page's social media stats, go to the Pages dashboard and switch to the Social media tab. Select the pages you'd like to get the stats for (use Ctrl + A to select all) and click Update factors. In the window that pops up, select the social media factors you want updated (under the Social media folder).

In a few moments, you'll see your pages' popularity in each social network you selected in the respective columns in your workspace.

To check Moz's Page Authority for your site's pages in WebSite Auditor, go to Preferences > Moz API Settings. Click Add Key if you already have a Moz API key; if you don't, click on the link to get a free one. Back in the Pages dashboard, select your pages and click on Update Factors; in the window that pops up, hit Add more ranking factors.

Here, find the External factors folder and expand it. Select Page Authority (PA) and click OK.

Now, hit Next to proceed with the update. You may want to deselect all other factors so that the check takes less time.

When the update is complete, right-click the header of any column to enter the workspace editing mode. Here, double-click Page Authority in the list of available columns on the left to add it to visible columns, and click OK. Scroll to the right to locate the column you just added and see your pages's authority according to Moz. To sort your pages by their PA, simply click on the header of the column.

Go to the Content Analysis module and select a page to analyze. At the next steps, enter your keywords and select the search engine you're optimizing the page for (this search engine will be used to look for competitors in).

In a moment, you'll see a detailed on-page analysis of your landing page. In the top left corner, you'll see the page's overall optimization score for your keywords. In the Content Optimization section of factors on the left, you can review your page's meta tags and keyword usage in each page element. As usual, you can click on any factor for details and SEO recommendations.

To audit another page, simply click the Add Page button and select one more page for analysis.

In WebSite Auditor, you can see how the top 10 ranking competitors are optimizing each page element for your keywords. To do this, go to the Content Analysis module. If your landing page hasn't been analyzed yet, select a page and enter your keywords. Next, click on any of the optimization factors on the left and switch to the Competitors tab. Here, you'll see how competitors are handling the factor in question: keyword usage in titles, meta descriptions, and headings; content length; keywords in links' anchor text, etc.

To see if your page speed can be improved, go to the Content Analysis > Page Audit module. If your landing page hasn't been analyzed yet, select a page and enter your keywords. Next, switch to the Technical factors in the left section and scroll down to the Page speed (Desktop). Here, you'll see a bunch of factors that Google uses to evaluate page speed — page size, server response time, number of redirects, uncompressed images and resources, etc. Ideally, you should aim to have all factors under this section marked with a green Correct sign.

If any of the factors under the Page speed (Desktop) section in your page audit have an Error, Warning, or Info status, that means your page speed can be improved. Click through the problematic factors and switch to Recommendation to get how-to-fix advice. If uncomressed images are found on your site, the Recommendation tab will also contain a download link with a minified version of your images. Download the archive and replace the heavy images on the page with their lighter versions.

To check if your page passes Google's mobile friendly test, go to the Content Analysis > Page Audit module. If your landing page hasn't been analyzed yet, select a page and enter your keywords. Next, switch to the Technical factors tab and scroll down to the Page usability (Mobile) section. The Mobile friendly factor will show whether or not the page is considered mobile friendly by Google. Use the factors below to find out if there are any usability problems with viewing the page on mobile devices, and, if so, how those can be fixed.

To view the mobile version of your page, click on the Mobile friendly factor under the Page usability (Mobile) section in the Content Analysis module. On the right, you'll see how your page is seen by Google's smartphone bot and users viewing your site on mobile devices.

Go to Content Analysis and switch to the TF-IDF dashboard — here you'll find the full list of topic-relevant terms and phrases associated with your target keywords, based on your top-ranking competitors' content. These are the terms and concepts that will help you increase the relevance of your pages in the semantic search era.

For optimization advice tailored for your specific page, check the Recommendations column. Look for the words and phrases with Use less or Use more recommendations to see how you can improve the relevance of your content. An Add recommendation will appear next to the terms most of your competitors are using, while you aren't.

You can edit and optimize your pages right in WebSite Auditor's Content Analysis module. Switch to Content Editor and starting editing your content in the WYSIWYG editor in the right-hand part of your screen. As you make changes, the on-page factors and your optimization score on the left will be recalculated in real time. When you're done editing, click Save page to save the upload-ready HTML to your hard drive.

When you're optimizing a page, a WYSIWYG editor is often not enough — e.g., if you need to add or remove some heading tags, rewrite your title, fix alt texts, etc. To view and edit the source code of your page, go to Content Editor and click the Edit source button in the right-hand part of your screen. As you make changes to your HTML, the on-page factors and your optimization score on the left will be recalculated in real time. When you're done editing, click Save page to save the optimized HTML to your hard drive.

In the Content Editor dashboard, switch to Title & Meta tags in the right-hand part of your screen to edit your title and description. As you type, you'll see a real-time preview of your page's Google snippet right below. When you're done editing your meta tags, click Save page to save the HTML to your hard drive, ready for upload.

You can run a quick domain overview report with the major SEO factors on the site you're analyzing (backlinks, search engine indexation, domain age, etc.) in the Domain Strength module in WebSite Auditor. All you need to do is switch to the module, and the report will be compiled automatically right away. You can edit the data included in the overview by clicking on the Settings button in the top right corner, and export the report in PDF by clicking on the Export button next to it. Mind that the PDF export is only available in the Enterprise version of the tool.

To genereate a site audit report, go to the Reports module in WebSite Auditor and select the type of report you're about to create. Click the pencil button next to the name of the report template to edit the data points it includes in the drag-n-drop editor, customize the report's look, or brand it with your logo.

You can then print the report, save it, upload it to the Web, or get a shareable report link to send to clients or colleagues. Mind that these export options (except for printing) are only available in the tool's Enterprise version.

To genereate a page optimization report, go to Content Analysis > Reports in WebSite Auditor and select the type of report you're about to create (or create a custom report template by clicking the + button in the top right corner). You can edit any report template by clicking on the pencil button next to its name, as well as customize the report's look and brand it with your logo.

If you have an Enterprise license of the tool, you'll be able to export the report by getting a shareable link, saving it to your hard drive, publishing it to the Web, etc.