diff --git a/README.md b/README.md index d299c76..d131444 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,8 @@ [![build status](https://secure.travis-ci.org/PuerkitoBio/goquery.png)](http://travis-ci.org/PuerkitoBio/goquery) +[![GoDoc](https://godoc.org/github.com/PuerkitoBio/goquery?status.png)](http://godoc.org/github.com/PuerkitoBio/goquery) + GoQuery brings a syntax and a set of features similar to [jQuery][] to the [Go language][go]. It is based on Go's [net/html package][html] and the CSS Selector library [cascadia][]. Since the net/html parser returns tokens (nodes), and not a full-featured DOM object, jQuery's manipulation and modification functions have been left off (no point in modifying data in the parsed tree of the HTML, it has no effect). Also, because the net/html parser requires UTF-8 encoding, so does goquery: it is the caller's responsibility to ensure that the source document provides UTF-8 encoded HTML. diff --git a/doc/tips.md b/doc/tips.md new file mode 100644 index 0000000..159c178 --- /dev/null +++ b/doc/tips.md @@ -0,0 +1,64 @@ +# Tips and tricks + +## Handle Non-UTF8 html Pages + +The `go.net/html` package used by `goquery` requires that the html document is UTF-8 encoded. When you know the encoding of the html page is not UTF-8, you can use the `iconv` package to convert it to UTF-8 (there are various implementation of the `iconv` API, see godoc.org for other options): + +``` +$ go get -u github.com/djimenez/iconv-go +``` + +and then: + +``` +// Load the URL +res, err := http.Get(url) +if e != nil { + // handle error +} +defer res.Body.Close() + +// Convert the designated charset HTML to utf-8 encoded HTML. +// `charset` being one of the charsets known by the iconv package. +utfBody, err := iconv.NewReader(res.Body, charset, "utf-8") +if err != nil { + // handler error +} + +// use utfBody using goquery +doc, err := goquery.NewDocumentFromReader(utfBody) +if err != nil { + // handler error +} +// use doc... +``` + +Thanks to github user @YuheiNakasaka. + +## Handle Javascript-based Pages + +`goquery` is great to handle normal html pages, but when most of the page is build dynamically using javascript, there's not much it can do. There are various options when faced with this problem: + +* Use a headless browser such as [webloop][]. +* Use a Go javascript parser package, such as [otto][]. + +You can find a code example using `otto` [in this gist][exotto]. Thanks to github user @cryptix. + +## For Loop + +If all you need is a normal `for` loop over all nodes in the current selection, where `Map` is not necessary, you can use the following: + +``` +sel := Doc().Find(".selector") +for i := range sel.Nodes { + single := sel.Eq(i) + // use `single` as a selection of 1 node +} +``` + +Thanks to github user @jmoiron. + +[webloop]: https://github.com/sourcegraph/webloop +[otto]: https://github.com/robertkrimen/otto +[exotto]: https://gist.github.com/cryptix/87127f76a94183747b53 +