Web Scraping for Marketing and Sales: Market Analysis with Netpeak Spider and Netpeak Checker

Use Cases
17Like
Comments
Share
Web Scraping for Marketing and Sales: Market Analysis with Netpeak Spider and Netpeak Checker

Search results are an endless source of valuable information that you can collect, filter and use for solving different tasks. You can use SERP data to both personal and professional purposes, including market analysis. In this article, I'm going to show you how to collect the list of key players in the niche, get their contact emails, and scrape the main information from Facebook business pages.

1. SERP Scraping

To perform an initial market analysis, you should scrape search results for your target queries. At first, collect all main keywords, and then use them for a SERP scraping. SERP scraping includes following steps:

  1. Launch Netpeak Checker and open the 'SE Scraper' window.
  2. Go to the dialogue box on the left side of the window and paste a list of queries you've collected.

    Uploading keywords in Netpeak Checker
  3. Switch the tab and go to the 'Settings'. Select the search engine, number of results, and scrape snippet types, and then click on the 'Search engine' button in the 'Additional settings' section.
    SERP scraping settings in Netpeak Checker
  4. In the opened window, choose the target country and language, and save your settings to get back into the scraper window. Push the 'Start' button.

    Search engine settings in Netpeak Checker
  5. Set the filter to remove all irrelevant websites from the list including price comparison websites and marketplaces like eBay, Amazon, etc.

    Filter settings in Netpeak Checker
  6. Click on the 'Transfer hosts' button to move domains to the main Netpeak Checker window.

    Transfer hosts button in Netpeak Checker

2. Collecting Contact Information and Social Media Links

After you've transferred hosts to the main table, you can scrape social media links (Facebook, Twitter, Instagram, Facebook, Linkedin, Youtube, and Pinterest) and emails for the further contacting with the websites owners. You can scrape all mentioned data, but in our case, there will be only emails and Facebook business pages links. Scraping includes several simple steps:

  1. Select all the types of data you want to scrape on 'Parameters' tab in a sidebar. We also recommend you to check the 'Title' box to detect and remove irrelevant pages from the results table.
    Choosing relevant information for scraping in 'Parameters' tab in Netpeak Checker
  2. When the analysis is finished, select all irrelevant URLs and remove them from a list with a 'Delete URLs' command in a shortcut menu.

    Cleaning results after scraping
  3. Export the final version of the results table as .xls or .csv file.

3. Scraping Data from Social Media Business Pages

Let's imagine that we should analyze the current level of social media activities and get the key information from the business pages for the list of companies. You can solve this task easily with the spider seo tool 'Scraping' feature. For example, you can extract basic information from the Facebook business page in several steps:

  1. Open any business page you've found before and inspect the source code of the 'Community' and 'About' sections.

    Inspect source code of Facebook business page
  2. Copy the class of container with the information you need (it will be highlighted).

    Class of container you need to scrape
  3. Launch Netpeak Spider and go to the 'Settings' → 'Scraping' and activate HTML scraping. Set the 'CSS Selector' type of search and 'Inner text' extraction mode. Then paste the container class in a search box and put a div. prefix right before it.

    Setting scraping of Facebook business pages in Netpeak Spider
  4. Open the 'Parameters' tab in a sidebar and untick all boxes except 'Scraping.'
  5. Upload a list of FB business accounts using the most convenient option from the 'List of URLs' menu. Save the settings and start crawling.
    Uploading Facebook business account in Netpeak Spider
  6. After scraping is finished, open the 'Reports' → 'Scraping' tab and click on 'All Results' button to see the table with all the data that crawler has found (a number of followers and likes, contact details, price range, etc.). Then, you can export them and save as a file on your PC.

    Scraping results of Facebook business accounts in Netpeak Spider

In a Nutshell

Data scraping is a good solution when you need to collect a big amount of information about a niche with a lot of players. It allows you to collect a lot of information including contact details and social media data that are in public access. You can use Netpeak Checker with a built-in SERP scraper tool and social links/emails extraction feature, and Netpeak Spider with a custom scraping option.

Netpeak Spider scrapes data from Twitter
Netpeak Spider scrapes data from Twitter

The data extracting method described above is not limited by one single use case.

Depending on your purposes, you can extract data from the most popular social media including Facebook, Twitter, and many others, and also set the custom search of any contact information.

This data can be used by a wide range of professionals – from marketers who analyze different niches and competitors to the sales specialists selling content marketing or social media promotion services.

All market data obtained with the Netpeak Spider and Netpeak Checker scraping feature can be combined into one summary table that different groups of specialists can use.

Final table with scraping results