* Created separate toggles for output-http-stats and output-tcp-stats
* Created HTTPWorkers option (defaults to 10) to allow larger number of HTTPWorkers to be spawned.
* Refactored output_http to allow dynamic http worker scaling when httpworkers equals -1
* Updated README.me to include updates and some common FAQs.
Can filter output_http by percentage of requests, constistent with a specific header value (FNV32-a hash-based)
Can filter output_http to those with header matching a specific regexp
Can filter output_http to those whose URL matches a specfic regexp
The error `RedirectNotAllowed` is wrapped by `url.Error`, causing it to not
currently be caught/ignored correctly for responses that issue redirects:
```
2014/01/06 12:17:17 Request error: Get https://example.com/foo: Redirects not allowed
```
It seems that two type assertions/conditions are required to unravel the
exact error. I can't see any way to write a test for this functionality --
the call to `defer()` or `log.Println()`.
Add a new CLI flag to specify which HTTP methods should be replayed by
HTTPOutput. If specified, any requests not matching will be dropped.
This can be useful for replaying requests against stateful environments.
e.g. You may not want to reproduce POST requests against an application if
it results in additional calls to the outside world.
Some variations I considered:
- Filtering on input instead of output. However we don't currently do any
request parsing on input, and doing so would likely have an impact on
performance.
- A request filtering plugin. We might want to revisit this if we add
anymore options to HTTPOutput. It could also be used to modify requests in
an existing file capture. Although because all plugins expect byte slices
we'd potentially have to parse requests more than once.
I don't think this warrants a separate test for HTTPOutput yet. But, again,
if we add anymore then we should split them out, and DRY up if possible.
Because we don't want to overload the scenarios covered by that one test.
At the time `CopyMulty()` calls `HTTPOutput.Write(data)` the contents of the
byte slice `data` is correct/consistent. However by the time the goroutine
for `HTTPOutput.sendRequest(data)` is scheduled, the contents of `data` has
changed, which in the case of our tests results in two things happening:
- The same request gets repeated many times.
- A request with the length of `EmitGET()` is made but with the larger
contents of `EmitPOST()`, causing it to be truncated and
`HTTPOutput.ParseRequest()` fails.
As I understand it, this is because the slice header of `data` is passed by
value into the goroutine, but the contents referred to by that header are
pointers which subsequently get overwritten.
By taking a `copy()` of the request data into a new buffer variable and
passing that to the goroutine, we can ensure that it doesn't get modified
in-flight.
Leonid Bugaev