Add a new CLI flag to specify which HTTP methods should be replayed by
HTTPOutput. If specified, any requests not matching will be dropped.
This can be useful for replaying requests against stateful environments.
e.g. You may not want to reproduce POST requests against an application if
it results in additional calls to the outside world.
Some variations I considered:
- Filtering on input instead of output. However we don't currently do any
request parsing on input, and doing so would likely have an impact on
performance.
- A request filtering plugin. We might want to revisit this if we add
anymore options to HTTPOutput. It could also be used to modify requests in
an existing file capture. Although because all plugins expect byte slices
we'd potentially have to parse requests more than once.
I don't think this warrants a separate test for HTTPOutput yet. But, again,
if we add anymore then we should split them out, and DRY up if possible.
Because we don't want to overload the scenarios covered by that one test.
At the time `CopyMulty()` calls `HTTPOutput.Write(data)` the contents of the
byte slice `data` is correct/consistent. However by the time the goroutine
for `HTTPOutput.sendRequest(data)` is scheduled, the contents of `data` has
changed, which in the case of our tests results in two things happening:
- The same request gets repeated many times.
- A request with the length of `EmitGET()` is made but with the larger
contents of `EmitPOST()`, causing it to be truncated and
`HTTPOutput.ParseRequest()` fails.
As I understand it, this is because the slice header of `data` is passed by
value into the goroutine, but the contents referred to by that header are
pointers which subsequently get overwritten.
By taking a `copy()` of the request data into a new buffer variable and
passing that to the goroutine, we can ensure that it doesn't get modified
in-flight.
Swap the order of calls to `EmitGET()` and `EmitPOST()` for HTTP output
tests in order to surface the problems:
- GET requests appear to be repeated. For three iterations, 5 GETs and only
1 POST request can be observed at `StartHTTP()`.
- POST requests are truncated somewhere along the line and cannot be parsed
by `HTTPOutput.ParseRequest()`. Resulting in:
Can not parse request POST /pub/WWW/ HTT malformed HTTP version "HTT"
There is an underlying problem with the test that it only checks the request
count, not the type/content of "good" requests.
The PSH flag tells the receiver to flush it's buffer. While this will
never be set for packets that have no data it can be, and often is, not
set for packets that do. For example, when a message spans more than one
segment the earlier segments often do not have the PSH flag set.
This alternative solution checks that the buffer is larger than the TCP
header.
Joseph Lawson