colly

# Colly

Lightning Fast and Elegant Scraping Framework for Gophers

Colly provides a clean interface to write any kind of crawler/scraper/spider.

With Colly you can easily extract structured data from websites, which can be used for a wide range of applications, like data mining, data processing or archiving.

[![GoDoc](https://godoc.org/github.com/gocolly/colly?status.svg)](https://pkg.go.dev/github.com/gocolly/colly/v2)
[![Backers on Open Collective](https://opencollective.com/colly/backers/badge.svg)](#backers) [![Sponsors on Open Collective](https://opencollective.com/colly/sponsors/badge.svg)](#sponsors) [![build status](https://github.com/gocolly/colly/actions/workflows/ci.yml/badge.svg)](https://github.com/gocolly/colly/actions/workflows/ci.yml)
[![report card](https://img.shields.io/badge/report%20card-a%2B-ff3333.svg?style=flat-square)](http://goreportcard.com/report/gocolly/colly)
[![view examples](https://img.shields.io/badge/learn%20by-examples-0077b3.svg?style=flat-square)](https://github.com/gocolly/colly/tree/master/_examples)
[![Code Coverage](https://img.shields.io/codecov/c/github/gocolly/colly/master.svg)](https://codecov.io/github/gocolly/colly?branch=master)
[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fgocolly%2Fcolly.svg?type=shield)](https://app.fossa.io/projects/git%2Bgithub.com%2Fgocolly%2Fcolly?ref=badge_shield)
[![Twitter URL](https://img.shields.io/badge/twitter-follow-green.svg)](https://twitter.com/gocolly)


## Sponsors

### [NodeMaven](https://go.nodemaven.com/CollyreadmeSeptember): The most efficient proxy provider for Web Scrapping and Automation with the Highest Quality IP on the market.

Why [NodeMaven](https://go.nodemaven.com/CollyreadmeSeptember)?
- 99.9% uptime
- ZIP Targeting
- IP filtering: all proxies have fraud score <97%
- No KYC required
- Unique free tools: Proxy Bandwidth Checker, Meta Tag Checker, IP Lookup and others!

Special codes for Colly users:
- **COLLY35** - 35% off to Mobile and Residential Proxies
- **COLLY40** - 40% off to ISP (Static) Proxies

[![NodeMaven](assets/nodemaven.png)](https://go.nodemaven.com/CollyreadmeSeptember)


### [SerpApi](https://serpapi.com/colly): Web Search API for your AI apps

Access real-time data from search engines such as Google, Bing, YouTube, Amazon, and more in structured JSON or Markdown results via a simple API

[![SerpApi](assets/serpapi.png)](https://serpapi.com/colly)


### [SX.ORG](https://sx.org/?c=COLLY): High-Speed High-Trust Proxies Built for Scraping & Automation at Any Scale

- 240+ Locations to choose from
- 120M+ IPs - Clean & Frequently Refreshed
- All proxy types: mobile, residential, data-center
- Extremely High-Trust - secure from bans
- Flexible pricing –pay-as-you-goor unlimited trafficoptions

Free trialfor new users - use promocode **COLLY** to claim 3GB of proxy

[![SX.ORG](assets/sx.jpg)](https://sx.org/?c=COLLY)

## Features

-   Clean API
-   Fast (>1k request/sec on a single core)
-   Manages request de