Benchmarking, packaging, and fix issues, tests and perfomance (#797)

### performance
- handling of the very big packet(any size that can be buffered)
- speeding up TCP sessions by using message hints: Added **proto.HasFullPayload** that helps to validate the entire HTTP request, it supports `Chunked` encoding too! Added **proto.HasRequestTitle** and **proto.HasResponseTitle** for validating the beginning of HTTP request. Those methods are used `input_raw.go` with `TCP`.
- supports Keep-Alive: the above functions helps to support keep-alive

### Packaging
- **capture:** engines(capture/doc.go)
- **tcp:** tcp message parser (tcp/doc.go)

### benchmarking
- **capture.BenchmarkPcapDump:** the benchmarks regarding dumping packets in a pcap file
- **capture.BenchmarkPcapFile:** the benchmarks of reading packets from a pcap file
- **capture.BenchmarkPcap:** the benchmarks of parsing packets from the loopback interface with pcap handles
- **proto.BenchmarkHasFullPayload:**: benchmarking this function which validates the HTTP payload
- **tcp.BenchmarkPacketParseAndSort:** benchmarks of parsing and sorting packets
- **tcp.BenchmarkMessageParserWithoutHint:** benchmarks of message reasembling by using `SYN` and `FIN` flag
- **tcp.BenchmarkMessageParserWithHint:** benchmarks of message reasembling by using `proto.HasRequestTitle` and `proto.HasFullPayload` flag

### issues
see linked issues

###  tests
- fixed input raw and engine tests

**Most of the changed of the files, was about using functionalities of** `tcp` **and** `capture` **in existing functionalities**
This commit is contained in:
Urban Ishimwe
2020-08-11 12:44:53 +03:00
committed by GitHub
parent 2f81fba370
commit fdc8b094f0
60 changed files with 3106 additions and 3825 deletions
+6 -4
View File
@@ -4,13 +4,15 @@ import (
"io"
"sync/atomic"
"time"
"github.com/buger/goreplay/size"
)
// BinaryOutputConfig struct for holding binary output configuration
type BinaryOutputConfig struct {
Workers int `json:"output-binary-workers"`
Timeout time.Duration `json:"output-binary-timeout"`
BufferSize int `json:"output-tcp-response-buffer"`
BufferSize size.Size `json:"output-tcp-response-buffer"`
Debug bool `json:"output-binary-debug"`
TrackResponses bool `json:"output-binary-track-response"`
}
@@ -78,7 +80,7 @@ func (o *BinaryOutput) startWorker() {
client := NewTCPClient(o.address, &TCPClientConfig{
Debug: o.config.Debug,
Timeout: o.config.Timeout,
ResponseBufferSize: o.config.BufferSize,
ResponseBufferSize: int(o.config.BufferSize),
})
deathCount := 0
@@ -135,7 +137,7 @@ func (o *BinaryOutput) Write(data []byte) (n int, err error) {
func (o *BinaryOutput) Read(data []byte) (int, error) {
resp := <-o.responses
Debug("[OUTPUT-TCP] Received response:", string(resp.payload))
Debug(2, "[OUTPUT-TCP] Received response:", string(resp.payload))
header := payloadHeader(ReplayedResponsePayload, resp.uuid, resp.startedAt, resp.roundTripTime)
copy(data[0:len(header)], header)
@@ -163,7 +165,7 @@ func (o *BinaryOutput) sendRequest(client *TCPClient, request []byte) {
stop := time.Now()
if err != nil {
Debug("Request error:", err)
Debug(1, "Request error:", err)
}
if o.config.TrackResponses {