Commit Graph
101 Commits
Author SHA1 Message Date
Urban IshimweandGitHub 5d8ca525a4 deprecate output-http non-compatible clients (#833)
for easy readability check changes by their underlying commits!

these benchmarks address the whole operation of the request cycle in goreplay.

_**goos: linux
goarch: amd64**_

**Using Compatible client:**

```
BenchmarkHTTPOutput-4      	   10417	    118969 ns/op	   12172 B/op	      93 allocs/op
BenchmarkHTTPOutputTLS-4   	    9136	    132929 ns/op	   12448 B/op	      97 allocs/op
```

**Using non-compatible client**
```
BenchmarkHTTPOutput-4      	     859	   1175040 ns/op	   15598 B/op	      46 allocs/op
BenchmarkHTTPOutputTLS-4   	     880	   1189643 ns/op	   15544 B/op	      52 allocs/op

```
Binary size reduced: **7%**

from these benchmarks, we may trade allocations with performance and memory!
2020-10-13 08:36:16 +03:00
Urban IshimweandGitHub fdc8b094f0 Benchmarking, packaging, and fix issues, tests and perfomance (#797)
### performance
- handling of the very big packet(any size that can be buffered)
- speeding up TCP sessions by using message hints: Added **proto.HasFullPayload** that helps to validate the entire HTTP request, it supports `Chunked` encoding too! Added **proto.HasRequestTitle** and **proto.HasResponseTitle** for validating the beginning of HTTP request. Those methods are used `input_raw.go` with `TCP`.
- supports Keep-Alive: the above functions helps to support keep-alive

### Packaging
- **capture:** engines(capture/doc.go)
- **tcp:** tcp message parser (tcp/doc.go)

### benchmarking
- **capture.BenchmarkPcapDump:** the benchmarks regarding dumping packets in a pcap file
- **capture.BenchmarkPcapFile:** the benchmarks of reading packets from a pcap file
- **capture.BenchmarkPcap:** the benchmarks of parsing packets from the loopback interface with pcap handles
- **proto.BenchmarkHasFullPayload:**: benchmarking this function which validates the HTTP payload
- **tcp.BenchmarkPacketParseAndSort:** benchmarks of parsing and sorting packets
- **tcp.BenchmarkMessageParserWithoutHint:** benchmarks of message reasembling by using `SYN` and `FIN` flag
- **tcp.BenchmarkMessageParserWithHint:** benchmarks of message reasembling by using `proto.HasRequestTitle` and `proto.HasFullPayload` flag

### issues
see linked issues

###  tests
- fixed input raw and engine tests

**Most of the changed of the files, was about using functionalities of** `tcp` **and** `capture` **in existing functionalities**
2020-08-11 12:44:53 +03:00
arijitad 01b6f1129c Restructre AppSetting and address comments. 2020-07-23 10:00:16 +05:30
arijitad 80b19f918b Map Appsetting to flag names. 2020-07-15 00:14:38 +05:30
Urban Ishimwe 39b1f80457 Merge branch 'master' into pro-merge 2020-06-16 22:43:08 +02:00
Arijit Das e9bbc252e0 Fix input raw test and go fmt. 2020-06-11 20:34:09 +05:30
Leonid Bugaev d67bc13db3 Merging PRO functionality back to OSS repo 2020-06-08 14:29:32 +03:00
Leonid Bugaev fa56e7644c More verbose S3 logging 2020-03-06 18:04:16 +03:00
Leonid Bugaev 7300b83894 Merge remote-tracking branch 'upstream/master' 2019-03-29 22:50:04 +07:00
Leonid Bugaev c48d07d2c6 Merge remote-tracking branch 'upstream/master' 2019-03-29 22:45:27 +07:00
Leonid Bugaev a8cfaa7581 Add --output-http-compatibility-mode
Add support for using default Go HTTP client
2019-03-10 10:29:28 +01:00
bruce34andLeonid Bugaev c693938ef9 Allow a range of output-http-workers with min and max. (#631)
This provides the ability to have a maximum number of workers, yet still allow scaling down. The scaling down is important to stop timeouts happening with under-utilised workers.
Note, although the code has workers_min, workers_max, the settings option --output-http-workers has remained and becomes the "max" value. I did this to hopefully not break too much for others. The min will default to 1, so users will see dynamic scaling unless they also set min to be the same. This will probably benefit them as they will avoid potential timeouts when the traffic reduces. But something to be aware of.
2019-02-16 09:32:56 +01:00
bruce34andLeonid Bugaev 267d6d0e2b Add adjustable output-http-stats-ms (#633)
This adds an option to log output-http-stats at a period desired.
2019-02-16 09:29:51 +01:00
bruce34andLeonid Bugaev e9a5cc19b5 Configurable http_output queue length (#634)
Makes sense!
2019-02-16 09:27:44 +01:00
Leonid Bugaev 0f8e2f2d52 Merge branch 'master' of https://github.com/buger/gor 2017-08-22 20:43:22 +05:00
thinkerouandLeonid Bugaev 89c34c6bf5 format file 2017-08-11 14:40:55 +03:00
Leonid Bugaev 74225ebb22 Rename all package imports from gor to goreplay 2017-05-01 17:56:50 +02:00
Leonid Bugaev ecb119e82a Merge branch 'master' of https://github.com/buger/gor 2017-02-07 13:41:47 +03:00
Leonid Bugaev 858dfaac78 Make HTTP output less verbose
It will act same with —debug flag
2017-02-07 13:33:45 +03:00
Leonid Bugaev 1a32de3dc6 Merge branch 'master' of https://github.com/buger/gor 2017-01-23 20:04:33 +03:00
Leonid BugaevandGitHub 8c16220532 Track http responses (#416) 2017-01-23 16:20:27 +04:00
Leonid Bugaev 92b02e4af4 Fmt 2016-12-23 19:56:33 +03:00
Leonid Bugaev 0b986a9c3c Merge branch 'master' of https://github.com/buger/gor 2016-08-31 18:23:27 +03:00
Leonid Bugaev a2f5e43375 Fix replayed response header 2016-08-31 16:27:07 +03:00
Leonid Bugaev 8e37136b8e Fix test (one more time) 2016-08-10 14:06:42 +03:00
Leonid BugaevandGitHub 7e245dfb18 Merge pull request #2 from buger/tcp-sessions
Add support for tcp sessions
2016-07-31 20:28:42 +03:00
Leonid Bugaev a4826fcff9 Add support for tcp sessions 2016-07-30 16:42:40 +03:00
Leonid Bugaev a6d263df1c Add support for binary input/output 2016-07-13 16:01:25 +03:00
Leonid Bugaev 37650df2c8 Refactor #308 2016-06-29 17:29:09 +03:00
Ricardo GalliandLeonid Bugaev 38535a2055 Added http client buffer size (#249) 2016-05-19 11:33:17 +06:00
pbsurf 5c272661e9 fix crash on missing payload metadata 2015-11-04 20:40:44 +00:00
Leonid Bugaev 661dd2026f Fix grouting spawn loop and memory leaks 2015-09-01 20:27:26 +03:00
Leonid Bugaev 929e027907 small formatting change 2015-08-23 13:08:51 +03:00
Leonid Bugaev f0b96af420 Add protection for malformed requests 2015-08-23 13:07:29 +03:00
Leonid Bugaev 16d7f23d98 New fixes 2015-08-22 09:41:22 +03:00
Leonid Bugaev 5d2f6eceea Fixes 2015-08-19 09:19:00 +03:00
Leonid Bugaev 3e763c43cf Unify all input/output plugins 2015-08-18 14:32:52 +03:00
Leonid Bugaev 211be0cddd Add request time and round-trip response time 2015-08-17 17:10:43 +03:00
Leonid Bugaev 70e7d8dd29 Reduce delay of raw input 2015-08-14 23:07:46 +03:00
Leonid Bugaev bb319b5fd2 Add request ID and token modifier middleware 2015-08-14 20:42:23 +03:00
Leonid Bugaev 0009555d96 Apply fmt 2015-08-11 11:08:14 +03:00
Leonid Bugaev 5098e388a2 Fix tests, now properly receive responses 2015-08-11 11:06:25 +03:00
Leonid Bugaev 9ab8b3c113 Add header to request and response 2015-08-10 22:57:18 +03:00
Leonid Bugaev a9be1811ea Merge branch 'master' into input-modifier
Conflicts:
	Makefile
2015-08-03 19:59:03 +03:00
Leonid Bugaev 961d5a754c Style fixes 2015-08-03 18:08:35 +03:00
Leonid Bugaev 083c97a9c4 Option to configure timeout 2015-07-31 22:39:01 +03:00
Leonid Bugaev a79e54c5f1 Do not set http header if it explicitly set 2015-07-31 21:41:06 +03:00
Leonid Bugaev 5cc4228dcd Merge branch 'master' into input-modifier
Conflicts:
	Dockerfile
	emitter.go
	http_client.go
	plugins.go
	proto/proto.go
2015-07-21 09:49:37 +05:00
Leonid Bugaev 28540f43a1 New portion of changes 2015-07-13 16:56:52 +05:00
Leonid Bugaev 8db14e334d Merge branch 'master' into input-modifier
Conflicts:
	Dockerfile
	Makefile
	output_http.go
	output_http_test.go
	settings.go
2015-07-09 19:58:24 +05:00