In some cases recorded files contain small amount malformed records.
The root cause is not clear, maybe multiple processes writing to the same file.
This change ensuer that file can be replayed, and malformed records will be ingored (with meaningful debug message showing exact line in the file with issue).
Additionally `--input-file-dry-run` mode was speed up a bit, since there were a few of Sleep statements which were not ignored when dry run is executed.
In some low traffic cases you can have cases when time between request minutes.
Additionally increasing speed can be not an option.
Now you can "skip" this pauses, by seetting max wait time
Now you can get information about file content without performing the actual replay.
For example, it can tell you how many requests in your files, and how long it will take to replay them.
Input file now pre-reads N requests, sort them by timestamp and emit on demand.
You can control read depth using --input-file-read-depth which is 100 by default.
It makes implementaiton faster, and it fix various issues when due to concurrenccy, or another issues requests gets addeed out of order.
In some rare cases, requests can be recorded out of order.
It was causing replay from file significanty slodown when such issue met.
Now it replays them but without adding sleep.
## Description
Hi 👋 I ran the [DeepSource](https://deepsource.io/) static analyzer on the forked copy of this repo and found some [interesting code quality issues](https://deepsource.io/gh/ankitdobhal/goreplay/issues/). This PR fixes a few of them.
## Summary of Fixes
- Added .deepsource.toml to fix bug risks
- Empty string test can be improved
- Nested if can be replaced with else-if
## Type of change
- [✔] Antipattern
- [ ] New feature (non-breaking change which adds functionality)
## Checklist:
- [✔] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
// PluginReader is an interface for input plugins
type PluginReader interface {
PluginRead() (msg *Message, err error)
}
// PluginWriter is an interface for output plugins
type PluginWriter interface {
PluginWrite(msg *Message) (n int, err error)
}
### performance
- handling of the very big packet(any size that can be buffered)
- speeding up TCP sessions by using message hints: Added **proto.HasFullPayload** that helps to validate the entire HTTP request, it supports `Chunked` encoding too! Added **proto.HasRequestTitle** and **proto.HasResponseTitle** for validating the beginning of HTTP request. Those methods are used `input_raw.go` with `TCP`.
- supports Keep-Alive: the above functions helps to support keep-alive
### Packaging
- **capture:** engines(capture/doc.go)
- **tcp:** tcp message parser (tcp/doc.go)
### benchmarking
- **capture.BenchmarkPcapDump:** the benchmarks regarding dumping packets in a pcap file
- **capture.BenchmarkPcapFile:** the benchmarks of reading packets from a pcap file
- **capture.BenchmarkPcap:** the benchmarks of parsing packets from the loopback interface with pcap handles
- **proto.BenchmarkHasFullPayload:**: benchmarking this function which validates the HTTP payload
- **tcp.BenchmarkPacketParseAndSort:** benchmarks of parsing and sorting packets
- **tcp.BenchmarkMessageParserWithoutHint:** benchmarks of message reasembling by using `SYN` and `FIN` flag
- **tcp.BenchmarkMessageParserWithHint:** benchmarks of message reasembling by using `proto.HasRequestTitle` and `proto.HasFullPayload` flag
### issues
see linked issues
### tests
- fixed input raw and engine tests
**Most of the changed of the files, was about using functionalities of** `tcp` **and** `capture` **in existing functionalities**
When Gor deals with multiple input files, it sorts them and replay their requests file after file.
This behavior is not convenient and should be changed. We should respect timestamps order when replaying requests.
