Attr was a 4-line wrapper around getAttributeValue, which itself was
a 6-line wrapper around getAttributePtr.
This commit inlines getAttributeValue into Attr and
have AttrOr delegate to Attr. Drops a layer of indirection and one
helper function; both methods get shorter and the empty-selection
check lives in one place.
winnow only ever reads sel.Nodes; the *Selection parameter was
incidental. Change its signature to winnow(nodes []*html.Node, ...)
and update its three callers (FilterMatcher, NotMatcher,
filterAndPush). filterAndPush no longer needs to fabricate a
throwaway Selection to satisfy the call.
While this change has o measurable runtime effect, it simplifies the code a bit,
by making winnow's signature clearer, and removes a now-useless line and its
associated comment in filterAndPush
parseHtml and parseHtmlWithContext duplicated the ParseFragment call,
the error check, and the panic message. parseHtml only differed by
passing a synthetic ElementNode as the context.
This commit simply delegates parseHtml to parseHtmlWithContext.
Move the "stop at first match" optimization out of HasMatcher and into
singleMatcher, where it can be reused elsewhere.
singleMatcher now implements MatchFirst, which calls the underlying
Matcher's MatchFirst if it provides one (cascadia-compiled matchers all
do), otherwise falls back to MatchAll and returns its first element.
singleMatcher.MatchAll wraps MatchFirst's result in a single-item slice
to respect the Matcher interface, so external SingleMatcher() users get
the slice signature while internal callers can avoid that allocation.
HasMatcher now constructs a singleMatcher directly to call MatchFirst
without a type assertion, removing the duplicated probe branch.
A Selection's Nodes field is exported, so callers can build a selection that
holds the same node more than once. Add a regression test asserting that
Children, ChildrenFiltered and Contents deduplicate the shared child set
rather than returning it once per duplicate source node.
Empty() collected the removed children into a nil slice grown via append,
forcing a series of grow-and-double reallocations proportional to the number
of children.
This commit walks the children once up front to count them, then allocate the
result slice at its final capacity before the removal loop.
The count pass is a cheap, branch-predicted pointer walk that replaces several
slice reallocations with a single allocation.
benchstat (-benchtime=2000x, count=10):
| old | new |
| allocs | allocs vs base |
Empty10-8 | 3.000 | 2.000 -33.33% |
Empty100-8 | 6.000 | 2.000 -66.67% |
Empty1000-8 | 9.000 | 2.000 -77.78% |
B/op: -47% / -56% / -53%
sec/op: Empty10 -53%, Empty100 -58%, Empty1000 ~ (noise)
HasMatcher only needs to know whether a child subtree contains a match,
but it tested len(m.MatchAll(c)) > 0 after wrapping the matcher with
SingleMatcher. SingleMatcher.MatchAll computes MatchFirst internally and
then wraps the result in a throwaway []*html.Node{node}, allocating a
one-element slice per matching child subtree just to check a boolean.
When the matcher exposes MatchFirst, probe it directly and test the returned
node against nil, avoiding the allocation entirely. This is the same trick
already used in SingleMatcher.MatchAll, just applied at the HasMatcher call
site so the intermediate slice is never built in the first place. The
SingleMatcher path is kept as a fallback for matchers that don't implement
MatchFirst.
benchstat (count=10):
| before | after |
| sec/op | sec/op vs base|
Has-8 | 2.675µ ± 17% | 2.208µ ± 17% -17.47%|
| before | after |
| B/op | B/op vs base |
Has-8 | 360.0 ± 0% | 240.0 ± 0% -33.33% |
| before | after |
|allocs/op | allocs/op vs base |
Has-8 | 20.00 ± 0% | 6.000 ± 0% -70.00% |
getParentNodes ran through mapNodes with a callback that allocated a
throwaway one-element []*html.Node{n.Parent} for every source node. Since
sibling nodes share the same parent, almost all of those slices were
immediately discarded as duplicates.
This commit collects the parents directly in a single loop, deduplicating via a
set as we go, so no per-node slice is allocated.
benchstat (count=10, all p=0.000):
| before | after |
| sec/op | sec/op vs base |
Parent-8 | 38.54µ ± 21%| 17.49µ ± 14% -54.63% |
ParentFiltered-8 | 46.57µ ± 13%| 20.91µ ± 24% -55.09% |
| before | after |
| B/op | B/op vs base |
Parent-8 | 13.24Ki ± 0%| 10.28Ki ± 0% -22.36% |
ParentFiltered-8 | 13.49Ki ± 0%| 10.58Ki ± 0% -21.60% |
| before | after |
| allocs/op | allocs/op vs base |
Parent-8 | 384.0 ± 0% | 10.00 ± 0% -97.40% |
ParentFiltered-8 | 392.0 ± 0% | 19.00 ± 0% -95.15% |
For the sibling types that collect every match (siblingAll,
siblingAllIncludingNonElements, siblingPrevAll, siblingNextAll), do a
cheap first-pass pointer walk to count the matches, then make the result
slice with that exact capacity. This removes the repeated append-driven
slice growth that dominated allocations on these hot traversal paths.
The Until cases are left untouched because counting would require running
the user predicate twice, and the single-result Next/Prev cases have
nothing to presize.
This drives Children/Contents/Siblings/Next*/Prev* through fewer
allocations:
allocs/op vs base B/op vs base
Siblings -57.23% -9.23%
SiblingsFiltered -54.49% -8.94%
NextAll -59.82% -10.04%
NextAllFiltered -55.83% -9.65%
PrevAll -55.32% -10.66%
PrevAllFiltered -52.53% -10.40%
ChildrenFiltered -15.38% -6.45%
Contents -7.69% -1.73%
geomean -25.07% -3.88%
The Until and single Next/Prev benchmarks are unchanged (bit-identical
allocs/op), confirming the count pre-pass only touches the collect-all
paths.
The quite verbose comment on top of the change is there so that future
generations won't waste time wondering why this weird loop is here.
BenchmarkAddNodesBig (DocW().Find("li") duplicated to ~1500 nodes,
then AddNodes onto an empty selection), -count=20:
sec/op B/op allocs/op
before 214.3µs ±12% 27.58Ki ±0% 24
after 104.8µs ±24% 45.24Ki ±0% 16
-51% +64% -33%
The B/op increase is the cost of presizing the map to its final
capacity in one shot rather than letting the runtime grow it
through smaller bucket arrays (which are freed but counted in
total bytes allocated). Peak resident memory is comparable.
Has(selector) and HasMatcher(m) used to call
s.document.Find(selector)/FindMatcher(m), materializing every matching
descendant in the entire document, then iterate s.Nodes x matchedNodes x
treeDepth via nodeContains to keep the elements that contain one.
This commit probes each selection node's subtree directly with SingleMatcher, which
short-circuits on the first descendant match.
BenchmarkHas (DocW().Find("h2").Has(".editsection")), -count=10:
sec/op B/op
before 61.97µs 744 B
after 3.33µs 360 B
-94.6% -51.6%
Three internal helpers build a map[*html.Node]bool from a known set of
nodes (or use one as a dedup accumulator over a known number of input
nodes) but allocate the map without a size hint. They then incur
grow-and-double rehashing as entries are inserted.
Pass len(nodes) as the size hint at make time:
- traversal.go mapNodes: dedup map shared by appendWithoutDuplicates
across every per-node result. mapNodes is the engine behind most
traversal helpers (Parents, Children, Next/Prev, Siblings, Find*,
Contents, ...), so the impact is broad.
- traversal.go ClosestNodes: target-node set built from nodes...
- filter.go winnowNodes: large-N path's lookup set built from nodes...
Double-digit performance gains across ~all benchmarks.
mapNodes always allocated a map[*html.Node]bool for deduplication, even
when called with a single source node where duplicates cannot arise.
Short-circuit to return the callback result directly when len(nodes)==1.
Also add a special case for when there are zero nodes.
This benefits all traversal methods (Find, Children, Parent, Next, Prev,
etc.) when operating on a single-node selection, which is the common
case after First(), Eq(), or directly on a Document.
name old ns/op new ns/op delta
Find-8 23200 14200 -38.8%
name old B/op new B/op delta
Find-8 4632 1496 -67.7%
name old allocs new allocs delta
Find-8 27 13 -51.9%
Replace the recursive closure in Selection.Text with a helper method call.
This keeps behavior unchanged while reducing runtime overhead in the hot path.
Benchmark (BenchmarkText, benchstat, n=12):
- sec/op: 2.133us -> 1.692us (-20.70%, p=0.000)
- B/op: 504 -> 504 (no change)
- allocs/op: 6 -> 6 (no change)
HasClass normalized every matching element's class attribute with
classTrimReplacer before checking for the target token, even when the
raw attribute value could not possibly contain that class at all.
Add a cheap strings.Contains(attr.Val, rawClass) pre-check and only run
the replacer on plausible matches. This preserves behavior for class
attributes containing tabs/newlines while skipping most of the work on
misses.
BenchmarkHasClass on arm64 over 10 runs:
name old time/op new time/op delta
HasClass-8 9.809µs 2.660µs -72.89% (p=0.000)
IsMatcher handled multi-node selections by calling m.Filter(s.Nodes) and
checking whether the returned slice was non-empty, forcing the matcher to walk
the whole selection and materialize all matches even though the API only needs
an existence check.
This commit replaces this with a simple loop over s.Nodes that calls m.Match(n)
and returns true on the first match. This also makes the single-node special
case unnecessary.
Benchmark results on arm64 over 10 runs:
name old time/op new time/op delta
Is-8 4.67µs 1.32µs -71.7%
IsPositional-8 21.44µs 0.82µs -96.2%
name old B/op new B/op delta
Is-8 96 80 -16.7%
IsPositional-8 1144 184 -83.9%
name old allocs new allocs delta
Is-8 5 4 -20.0%
IsPositional-8 11 7 -36.4%
HasNodes routed through FilterFunction -> winnowFunction -> grep, which
allocated a single-node Selection and a closure capture per source element,
just to call sel.Contains(n) which is itself a one-iteration sliceContains over
the wrapped node. This commit inlines a direct double loop over s.Nodes x nodes
calling nodeContains, and push the result via pushStack.
name old allocs/op new allocs/op delta
Has-8 55.00 ± 0% 24.00 ± 0% -56.36% (p=0.000)
HasNodes-8 752.00 ± 0% 4.00 ± 0% -99.47% (p=0.000)
HasSelection-8 752.00 ± 0% 3.00 ± 0% -99.60% (p=0.000)
In the same spirit as #545, replaces costly grep() with a loop.
While escape analysis managed to stack-allocate the per-node
Selection through grep's inlining, the closure capture itself (nodes and keep)
was still heap-allocated. Local benchmarks are showing a 30% allocs/op
reduction, and a ~10-20% reduction in sec/op.
dependabot[bot]