the current UUID generator runs at:
```
BenchmarkMessageUUID 13599342 87.7 ns/op 16 B/op 0 allocs/op
```
with the former version that was running on
```
BenchmarkMessageUUID 2818203 427 ns/op 96 B/op 2 allocs/op
```
it fixes#842#851
// PluginReader is an interface for input plugins
type PluginReader interface {
PluginRead() (msg *Message, err error)
}
// PluginWriter is an interface for output plugins
type PluginWriter interface {
PluginWrite(msg *Message) (n int, err error)
}
The focus here was to **reduce allocation in TCP parser** but speed may have hopeful improved too!
pool no longer use map's key of **string** it uses **uint64**
**Benchmarks** was revamped to be more clear
if you want to compare these results copy the benchmark in tcp/bench_test.go@reduce-allocation to tcp/bench_test.go@master:
**before(master)**:
```
BenchmarkPacketParseAndSort-4 1000000 1006 ns/op 64 B/op 2 allocs/op
BenchmarkMessageParserWithoutHint-4 625 1772309 ns/op 1000 packets/op 419096 B/op 10045 allocs/op
BenchmarkMessageParserWithHint-4 74 14969926 ns/op 1000 chunks/op 1002 packets/op 450992 B/op 10126 allocs/op
```
**After(this branch)**:
```
BenchmarkPacketParseAndSort-4 1267662 941 ns/op 64 B/op 2 allocs/op
BenchmarkMessageParserWithoutHint-4 2256 523474 ns/op 1000 packets/op 243530 B/op 1037 allocs/op
BenchmarkMessageParserWithHint-4 80 13990955 ns/op 1000 chunks/op 1002 packets/op 268609 B/op 1099 allocs/op
```
### performance
- handling of the very big packet(any size that can be buffered)
- speeding up TCP sessions by using message hints: Added **proto.HasFullPayload** that helps to validate the entire HTTP request, it supports `Chunked` encoding too! Added **proto.HasRequestTitle** and **proto.HasResponseTitle** for validating the beginning of HTTP request. Those methods are used `input_raw.go` with `TCP`.
- supports Keep-Alive: the above functions helps to support keep-alive
### Packaging
- **capture:** engines(capture/doc.go)
- **tcp:** tcp message parser (tcp/doc.go)
### benchmarking
- **capture.BenchmarkPcapDump:** the benchmarks regarding dumping packets in a pcap file
- **capture.BenchmarkPcapFile:** the benchmarks of reading packets from a pcap file
- **capture.BenchmarkPcap:** the benchmarks of parsing packets from the loopback interface with pcap handles
- **proto.BenchmarkHasFullPayload:**: benchmarking this function which validates the HTTP payload
- **tcp.BenchmarkPacketParseAndSort:** benchmarks of parsing and sorting packets
- **tcp.BenchmarkMessageParserWithoutHint:** benchmarks of message reasembling by using `SYN` and `FIN` flag
- **tcp.BenchmarkMessageParserWithHint:** benchmarks of message reasembling by using `proto.HasRequestTitle` and `proto.HasFullPayload` flag
### issues
see linked issues
### tests
- fixed input raw and engine tests
**Most of the changed of the files, was about using functionalities of** `tcp` **and** `capture` **in existing functionalities**
All flags that expect buffer as input i.e. `--output-file-size-limit`, `--output-file-max-size-limit`, `--copy-buffer-size` and `input-raw-buffer-size` can now parse inputs from differents bases and data units like: `10mb`, `10kb`, `100gb`, `18tb`, `11839023`....
data units and bases are case insensitive, the parser accepts only the format of [Go integer literals](https://golang.org/ref/spec#Integer_literals)
Added `—input-raw-buffer-size` - Controls size of the OS buffer (in
bytes) which holds packets until they dispatched. Default value depends
by system: in Linux around 2MB. If you see big package drop, increase
this value.
Additionally snaplen (max number of bytes being read for each packet)
now dynamically set based on interface MTU + max header size. In most
situations it should reduce package drop, because each packet will
consume less space in buffer.
Previously latency calcualted as Response.End - Request.Start
Where both End and Start is a last and first packets
This calcualtion is wrong, because it is total roundtrip
Correct server latency will be Response.End - Request.End
In addition added new `--input-raw-timestamp-type` option
which allows choose more precise packet timestamp source (if available).
Can be useful in case of non standard network interfaces when like
tunnels or SPAN ports, when IP of interface does not match with IP of
packet.
Can be used to read traffic from multiple ports and interfaces at once
as well.
Leonid Bugaev