Benchmarking, packaging, and fix issues, tests and perfomance (#797)

### performance
- handling of the very big packet(any size that can be buffered)
- speeding up TCP sessions by using message hints: Added **proto.HasFullPayload** that helps to validate the entire HTTP request, it supports `Chunked` encoding too! Added **proto.HasRequestTitle** and **proto.HasResponseTitle** for validating the beginning of HTTP request. Those methods are used `input_raw.go` with `TCP`.
- supports Keep-Alive: the above functions helps to support keep-alive

### Packaging
- **capture:** engines(capture/doc.go)
- **tcp:** tcp message parser (tcp/doc.go)

### benchmarking
- **capture.BenchmarkPcapDump:** the benchmarks regarding dumping packets in a pcap file
- **capture.BenchmarkPcapFile:** the benchmarks of reading packets from a pcap file
- **capture.BenchmarkPcap:** the benchmarks of parsing packets from the loopback interface with pcap handles
- **proto.BenchmarkHasFullPayload:**: benchmarking this function which validates the HTTP payload
- **tcp.BenchmarkPacketParseAndSort:** benchmarks of parsing and sorting packets
- **tcp.BenchmarkMessageParserWithoutHint:** benchmarks of message reasembling by using `SYN` and `FIN` flag
- **tcp.BenchmarkMessageParserWithHint:** benchmarks of message reasembling by using `proto.HasRequestTitle` and `proto.HasFullPayload` flag

### issues
see linked issues

###  tests
- fixed input raw and engine tests

**Most of the changed of the files, was about using functionalities of** `tcp` **and** `capture` **in existing functionalities**
This commit is contained in:
Urban Ishimwe
2020-08-11 12:44:53 +03:00
committed by GitHub
parent 2f81fba370
commit fdc8b094f0
60 changed files with 3106 additions and 3825 deletions
+8 -9
View File
@@ -1,10 +1,9 @@
package main
import (
"crypto/rand"
"encoding/base64"
"errors"
"fmt"
"math/rand"
"time"
)
@@ -40,16 +39,18 @@ func (i *TestInput) Read(data []byte) (int, error) {
}
return len(buf) + len(header), nil
case <-time.After(10 * time.Second):
return 0, fmt.Errorf("timed out waiting for read")
case <-i.stop:
return 0, ErrorStopped
}
}
// Close closes this plugin
func (i *TestInput) Close() error {
close(i.stop)
return nil
}
// EmitBytes sends data
func (i *TestInput) EmitBytes(data []byte) {
i.data <- data
}
@@ -77,8 +78,7 @@ func (i *TestInput) EmitLargePOST() {
rs := base64.URLEncoding.EncodeToString(rb)
i.data <- []byte("POST / HTTP/1.1\nHost: www.w3.org\nContent-Length:5242880\r\n\r\n" + rs)
Debug("Sent large POST")
i.data <- []byte("POST / HTTP/1.1\r\nHost: www.w3.org\nContent-Length:5242880\r\n\r\n" + rs)
}
// EmitSizedPOST emit a POST with a payload set to a supplied size
@@ -88,13 +88,12 @@ func (i *TestInput) EmitSizedPOST(payloadSize int) {
rs := base64.URLEncoding.EncodeToString(rb)
i.data <- []byte("POST / HTTP/1.1\nHost: www.w3.org\nContent-Length:5242880\r\n\r\n" + rs)
Debug("Sent large POST")
i.data <- []byte("POST / HTTP/1.1\r\nHost: www.w3.org\nContent-Length:5242880\r\n\r\n" + rs)
}
// EmitOPTIONS emits OPTIONS request, similar to GET
func (i *TestInput) EmitOPTIONS() {
i.data <- []byte("OPTIONS / HTTP/1.1\nHost: www.w3.org\r\n\r\n")
i.data <- []byte("OPTIONS / HTTP/1.1\r\nHost: www.w3.org\r\n\r\n")
}
func (i *TestInput) String() string {