LK_Scraper : An Fully Configurable LinkedIn Scrape

Lk_scraper is an fully configurable LinkedIn scrape : scrape anything within LinkedIn

Installation

$ pip install git+git://github.com/jqueguiner/lk_scraper

Setup

  • Using Docker compose

$ docker-compose up -d
$ docker-compose run lk_scraper python3

  • Using Docker only for selenium server

First, you need to run a selenium server

$ docker run -d -p 4444:4444 –shm-size 2g selenium/standalone-firefox:3.141.59-20200326

After running this command, from the browser navigate to your IP address followed by the port number and /grid/console. So the command will be http://localhost:4444/grid/console.

Also Read – Lollipopz : Data Exfiltration Utility For Testing Detection Capabilities

Retrieving Cookie

  • Browser-Independent:
  1. Navigate to Linkedin.com and log in
  2. Open up the browser developer tools (Ctrl-Shift-I or right click -> inspect element)
  • Chrome:
    • Select the Application tab
    • Under the Storage header on the left-hand menu, click the Cookies dropdown and select www.linkedin.com
    • Find the li_at cookie, and double click the value to select it before copying
  • Firefox:
    • Select Storage tab
    • Click the Cookies dropdown and select www.linkedin.com
    • Find and copy the li_at value

Setting Up The Cookie

  • Method 1 : Setting the cookie in the config file

You can add your LinkedIn li_at cookie in the config file that is located in your home (~/.lk_scraper/config.yml) see

  • Method 2 : Setting the cookie at the Scraper level

from lk_scraper import Scraper
li_at = “My_super_linkedin_cookie”
scraper = Scraper(li_at=li_at)

  • Method 3 : Using Variable Environment

(Not implemented Yet)

$ export LI_AT=”My_super_linkedin_cookie”

Example

run the jupyter notebook linkedin-example.ipynb

  • Usage

>>from lk_scraper import Scraper
>>scraper = Scraper()

  • Company Scraping

>>from lk_scraper import Scraper
>>scraper = Scraper()
>>company = scraper.get_object(object_name=’company’, object_id=’apple’)

  • Profile Scraping

>>from lk_scraper import Scraper
>>scraper = Scraper()
>>profil = scraper.get_object(object_name=’profil’, object_id=’jlqueguiner’)

R K

Recent Posts

WID_LoadLibrary : The Intricacies Of DLL Management In Windows

WID_LoadLibrary is a custom implementation inspired by the Windows API function LoadLibrary, which is used…

15 hours ago

Locksmith : A Tool For Securing Active Directory Certificate Services

Locksmith is a specialized tool designed to identify and remediate vulnerabilities in Active Directory Certificate…

15 hours ago

Uscrapper Vanta : A Cutting-Edge OSINT Tool For Advanced Data Extraction

Uscrapper Vanta is a powerful open-source intelligence (OSINT) tool designed to revolutionize web scraping and…

15 hours ago

Pake : Transforming Webpages Into Desktop Applications

Pake is an innovative tool designed to convert any webpage into a desktop application with…

19 hours ago

Bevy : Exploring The Frontier Of Game Development With Rust

Bevy is an open-source, data-driven game engine built in Rust, designed to simplify game development…

19 hours ago

AppFlowy Cloud : Enhancing Collaboration With Secure Cloud Infrastructure

AppFlowy Cloud is a robust component of the AppFlowy ecosystem, designed to provide secure user…

2 days ago