20 Commits
Author SHA1 Message Date
Zapan GaoandGitHub abbd249c49 Adding support for specify input kafka offset for each partitions (#1242)
Adding a new flag `--input-kafka-offset` for supporting consumes from Kafka by specified offset.
2024-04-09 11:55:03 +03:00
Dima GolomozyandGitHub 40946831b9 goreplay-cli package (#1148)
change package from `main -> goreplay`
this will allow importing `goreplay` as a package
2023-01-12 18:24:45 +03:00
choi seandGitHub 951cd72a92 Support SASL Protocol of Kafka (#1092)
* Support SASL Protocol of Kafka

* Add comments

* Fix Unused parameter detected in function RVV-B0012
2022-07-23 17:00:43 +03:00
Erik SchwellerandGitHub 09d7d42254 Address issues flagged by (#938) 2021-06-08 21:05:16 +03:00
Urban Ishimwe 6d812ceb7f changes plugins reader and writer method
// PluginReader is an interface for input plugins
type PluginReader interface {
	PluginRead() (msg *Message, err error)
}

// PluginWriter is an interface for output plugins
type PluginWriter interface {
	PluginWrite(msg *Message) (n int, err error)
}
2020-11-02 06:15:10 +02:00
David FradinandGitHub 358cb78a53 Fix #835: kafka bug (#836)
Correction of error introduce in PR #800
2020-10-19 10:56:07 +03:00
Urban IshimweandGitHub 5d8ca525a4 deprecate output-http non-compatible clients (#833)
for easy readability check changes by their underlying commits!

these benchmarks address the whole operation of the request cycle in goreplay.

_**goos: linux
goarch: amd64**_

**Using Compatible client:**

```
BenchmarkHTTPOutput-4      	   10417	    118969 ns/op	   12172 B/op	      93 allocs/op
BenchmarkHTTPOutputTLS-4   	    9136	    132929 ns/op	   12448 B/op	      97 allocs/op
```

**Using non-compatible client**
```
BenchmarkHTTPOutput-4      	     859	   1175040 ns/op	   15598 B/op	      46 allocs/op
BenchmarkHTTPOutputTLS-4   	     880	   1189643 ns/op	   15544 B/op	      52 allocs/op

```
Binary size reduced: **7%**

from these benchmarks, we may trade allocations with performance and memory!
2020-10-13 08:36:16 +03:00
David FradinandGitHub a2bbbb18ef Merge branch 'master' into feature/kafka_tls 2020-08-12 14:44:51 +02:00
Urban IshimweandGitHub fdc8b094f0 Benchmarking, packaging, and fix issues, tests and perfomance (#797)
### performance
- handling of the very big packet(any size that can be buffered)
- speeding up TCP sessions by using message hints: Added **proto.HasFullPayload** that helps to validate the entire HTTP request, it supports `Chunked` encoding too! Added **proto.HasRequestTitle** and **proto.HasResponseTitle** for validating the beginning of HTTP request. Those methods are used `input_raw.go` with `TCP`.
- supports Keep-Alive: the above functions helps to support keep-alive

### Packaging
- **capture:** engines(capture/doc.go)
- **tcp:** tcp message parser (tcp/doc.go)

### benchmarking
- **capture.BenchmarkPcapDump:** the benchmarks regarding dumping packets in a pcap file
- **capture.BenchmarkPcapFile:** the benchmarks of reading packets from a pcap file
- **capture.BenchmarkPcap:** the benchmarks of parsing packets from the loopback interface with pcap handles
- **proto.BenchmarkHasFullPayload:**: benchmarking this function which validates the HTTP payload
- **tcp.BenchmarkPacketParseAndSort:** benchmarks of parsing and sorting packets
- **tcp.BenchmarkMessageParserWithoutHint:** benchmarks of message reasembling by using `SYN` and `FIN` flag
- **tcp.BenchmarkMessageParserWithHint:** benchmarks of message reasembling by using `proto.HasRequestTitle` and `proto.HasFullPayload` flag

### issues
see linked issues

###  tests
- fixed input raw and engine tests

**Most of the changed of the files, was about using functionalities of** `tcp` **and** `capture` **in existing functionalities**
2020-08-11 12:44:53 +03:00
David FRADIN f97f2ba5f3 Add Mutual TLS config for Kafka input and ouput 2020-08-06 13:26:36 +02:00
arijitad 80b19f918b Map Appsetting to flag names. 2020-07-15 00:14:38 +05:30
Leonid Bugaev c5d1112e7a Add way to control packet capture buffer size and optimize snaplen
Added `—input-raw-buffer-size` - Controls size of the OS buffer (in
bytes) which holds packets until they dispatched. Default value depends
by system: in Linux around 2MB. If you see big package drop, increase
this value.

Additionally snaplen (max number of bytes being read for each packet)
now dynamically set based on interface MTU + max header size. In most
situations it should reduce package drop, because each packet will
consume less space in buffer.
2018-05-27 12:31:47 +03:00
UbuntuandLeonid Bugaev a65256d707 Fixed kafka client run of brokers and index out of range issues 2018-02-27 20:17:45 +03:00
田欧andLeonid Bugaev 17f6c59061 remove some else statements for reducing indent (#488)
* remove else

* remove else
2017-08-22 19:22:59 +05:00
田欧andLeonid Bugaev 16b6a3cc31 Adjust package order (#487)
* format package order

* format package order

* format package order

* format package order and format code by fmt
2017-08-22 19:15:01 +05:00
Nicholas WiersmaandLeonid Bugaev e726e45dc0 Fix kafka issue (#419) 2017-01-25 15:25:06 +04:00
Leonid Bugaev 5e90377a7f Fmt changes 2017-01-20 21:01:19 +03:00
Leonid Bugaev f520b338f3 Add tests and modify to work same as Kafka output 2017-01-20 21:00:23 +03:00
Nicholas Wiersma dfd5fa0c4f Update documentation 2016-12-22 10:28:35 +01:00
Nicholas Wiersma 1f19e550bb Create kafka input 2016-12-22 09:59:48 +01:00