Commit Graph
43 Commits
Author SHA1 Message Date
Urban Ishimwe 6d812ceb7f changes plugins reader and writer method
// PluginReader is an interface for input plugins
type PluginReader interface {
	PluginRead() (msg *Message, err error)
}

// PluginWriter is an interface for output plugins
type PluginWriter interface {
	PluginWrite(msg *Message) (n int, err error)
}
2020-11-02 06:15:10 +02:00
Urban IshimweandGitHub fdc8b094f0 Benchmarking, packaging, and fix issues, tests and perfomance (#797)
### performance
- handling of the very big packet(any size that can be buffered)
- speeding up TCP sessions by using message hints: Added **proto.HasFullPayload** that helps to validate the entire HTTP request, it supports `Chunked` encoding too! Added **proto.HasRequestTitle** and **proto.HasResponseTitle** for validating the beginning of HTTP request. Those methods are used `input_raw.go` with `TCP`.
- supports Keep-Alive: the above functions helps to support keep-alive

### Packaging
- **capture:** engines(capture/doc.go)
- **tcp:** tcp message parser (tcp/doc.go)

### benchmarking
- **capture.BenchmarkPcapDump:** the benchmarks regarding dumping packets in a pcap file
- **capture.BenchmarkPcapFile:** the benchmarks of reading packets from a pcap file
- **capture.BenchmarkPcap:** the benchmarks of parsing packets from the loopback interface with pcap handles
- **proto.BenchmarkHasFullPayload:**: benchmarking this function which validates the HTTP payload
- **tcp.BenchmarkPacketParseAndSort:** benchmarks of parsing and sorting packets
- **tcp.BenchmarkMessageParserWithoutHint:** benchmarks of message reasembling by using `SYN` and `FIN` flag
- **tcp.BenchmarkMessageParserWithHint:** benchmarks of message reasembling by using `proto.HasRequestTitle` and `proto.HasFullPayload` flag

### issues
see linked issues

###  tests
- fixed input raw and engine tests

**Most of the changed of the files, was about using functionalities of** `tcp` **and** `capture` **in existing functionalities**
2020-08-11 12:44:53 +03:00
Urban Ishimwe 39b1f80457 Merge branch 'master' into pro-merge 2020-06-16 22:43:08 +02:00
Arijit Das 73e26d48d9 Fix CI comments. 2020-06-11 19:24:38 +05:30
Arijit Das 1c87a339a5 Address comments. 2020-06-11 19:21:31 +05:30
Leonid Bugaev d67bc13db3 Merging PRO functionality back to OSS repo 2020-06-08 14:29:32 +03:00
Leonid Bugaev ab2ba8400d Improve S3 debugging 2020-06-06 17:07:51 +03:00
Arijit Das db28858804 Refactor emitter.go and fix test accordingly. 2020-06-06 16:00:43 +05:30
Arijit Das e1bf25682f Minor changes. 2020-06-03 22:41:58 +05:30
Arijit Das 039d7c3f26 Fix race input_file.go while closing the file and remove dead code. 2020-06-03 22:35:47 +05:30
Leonid Bugaev fa56e7644c More verbose S3 logging 2020-03-06 18:04:16 +03:00
Leonid Bugaev 962087065a S3 fixes 2019-01-22 23:08:23 +01:00
Leonid Bugaev 06ec570286 Merge remote-tracking branch 'upstream/master' 2017-07-20 21:09:04 +03:00
Leonid Bugaev b3dffae66f When using --input-file it should exit once file is fully read 2017-06-27 13:56:14 +02:00
Leonid Bugaev 2b6203c110 Fix AWS config 2017-04-29 11:39:01 +02:00
Leonid Bugaev 973b028517 Merge branch 'master' into s3 2017-01-23 13:27:56 +03:00
Leonid Bugaev 5a7bdd3d3a Add support for reading from s3 2016-12-23 19:56:25 +03:00
Leonid Bugaev ddf2dc2876 fixes 2016-12-19 20:16:11 +03:00
Leonid Bugaev 7c33336433 Add missing change 2016-06-29 17:29:36 +03:00
Leonid Bugaev 37650df2c8 Refactor #308 2016-06-29 17:29:09 +03:00
Yohan Legat 9e7351d28c Resolve buger/gor#300 : respect timestamps when replaying requests
When Gor deals with multiple input files, it sorts them and replay their requests file after file.
This behavior is not convenient and should be changed. We should respect timestamps order when replaying requests.
2016-06-17 11:53:10 +02:00
Marc LallaouretandLeonid Bugaev d0e1858eb3 Fix Close implementation of plugins (#305)
Indeed they do not declare error as return type and so, were not called at the end of the program
2016-06-16 22:33:07 +06:00
Leonid Bugaev a3aeca204c Check dst and src IP's for packets (#295)
* Add IP check

* Check dst and src ips
2016-06-09 18:41:01 +06:00
Leonid Bugaev cefa2b5cc2 Write files in chunks (#293)
* Chunked output strategy
* Fix sorting with indexes >= 10
2016-06-07 18:46:13 +06:00
Leonid Bugaev ede0fb4c4c Allow looping mode for --input-file 2016-06-01 18:49:20 +05:00
Leonid Bugaev 651bd82d3d input-file support for gzip files 2016-06-01 10:41:29 +05:00
Leonid Bugaev 0f5b509397 Add support for reading multiple files 2016-05-31 20:44:04 +05:00
Leonid Bugaev d40d6c6116 Fix --input-tcp for payloads > 64kb 2016-05-17 19:56:13 +05:00
Jon Barber e659f65d76 Replace the use of bufio.Scanner with bufio.Reader so that requests over 64Kb in size can be replayed from a capture file. 2016-04-21 10:15:48 +01:00
pbsurf 5c272661e9 fix crash on missing payload metadata 2015-11-04 20:40:44 +00:00
Leonid Bugaev 6d2a895db3 Fix file input replay speed 2015-09-03 21:50:32 +03:00
Leonid Bugaev 16d7f23d98 New fixes 2015-08-22 09:41:22 +03:00
Leonid Bugaev 3c45595a96 More race fixes 2015-08-20 09:27:44 +03:00
Leonid Bugaev b53fba0415 Fix formatting issues 2015-08-18 17:28:09 +03:00
Leonid Bugaev ae35f88004 Remove magic numbers 2015-08-18 17:18:02 +03:00
Leonid Bugaev 7790b33cea Refactor protocol scanner 2015-08-18 15:52:23 +03:00
Leonid Bugaev a971d8a843 Change file format to be text based 2015-08-18 15:48:42 +03:00
Leonid Bugaev 125d9db834 Next batch 2015-07-17 11:30:43 +05:00
Leonid Bugaev f2cf81c5bf FileInput should have own limiter algorithm 2014-10-26 15:00:00 +03:00
Sam MacPherson 1a7cf2bbab fixed file input sleep timer 2014-09-23 16:54:26 -04:00
Leonid Bugaev 0fb1d83284 Simplify directory structure 2014-03-16 22:24:27 +06:00
Leonid Bugaev 95d3676d83 Add delay between request for input_file 2013-10-29 20:49:59 +01:00
Leonid Bugaev 72952fddd2 Added input_file plugin 2013-10-29 18:53:35 +01:00