Dragline Documentation

Dragline Documentation

Release 0.6 beta Ashwin Rajeev

April 27, 2016

Contents

1 What is Dragline?

1

2 Features

3

3 First steps

5

3.1 Installation guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3.2 Dragline Tutorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

4 Contents

9

4.1 Request and Response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

4.2 HTML Parser Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4.3 Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4.4 Dragline shell Tutorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

5 Indices and tables

15

Python Module Index

17

i

ii

CHAPTER 1 What is Dragline?

Dragline is a pure pythonic library that intended to create the crawling applications in python.Builtin crawler is capable of scraping any no of pages and fetches the URL.The main target of dragline is to create custom spiders on the web. The creative usage of the Dragline remains in the hands of the developer.This documentation leads a developer to easily come up with all the tools available and how they can be used to build a custom spider in few minutes.

1

Dragline Documentation, Release 0.6 beta

2

Chapter 1. What is Dragline?

CHAPTER 2 Features

There are some crawling packages available in the community.Effieciency must be given the utmost priority since the emergence of large projects.Dragline is different from others in many aspects.The most significant feautures of dragline those makes it powerful:

1. Built from scratch. 2. redis backend for persistant storage.

3

Dragline Documentation, Release 0.6 beta

4

Chapter 2. Features

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download