Commit Graph
169 Commits
Author SHA1 Message Date
Minio Trusted bfb92a27b7 Update yaml files to latest version RELEASE.2020-12-23T02-24-12Z 2020-12-23 02:43:25 +00:00
HarshavardhanaandGitHub 5c451d1690 update x/net/http2 to address few bugs (#11144)
additionally also configure http2 healthcheck
values to quickly detect unstable connections
and let them timeout.

also use single transport for proxying requests
2020-12-21 21:42:38 -08:00
Anis ElleuchandGitHub 5434088c51 replication: Ensure to always use nano precision source modtime (#11135) 2020-12-18 11:37:28 -08:00
HarshavardhanaandGitHub 5df61ab96b fix: remove gorilla/rpc/ deps fully after our fork (#11108) 2020-12-15 11:18:06 -08:00
HarshavardhanaandGitHub 705e196b6c upgrade event notification dependencies (#11095) 2020-12-12 20:31:28 -08:00
HarshavardhanaandGitHub 4939987eb8 update deps to latest for some vulnerable deps (#11080)
- github.com/miekg/dns
- update elasticsearch library deps
  to circumvent some aws-sdk-go deps
2020-12-10 13:23:06 -08:00
HarshavardhanaandGitHub 12391ec4ba update nats.io/jwt dependency (#11066)
primarily for a vulnerability report https://nvd.nist.gov/vuln/detail/CVE-2020-26521
2020-12-09 14:30:35 -08:00
HarshavardhanaandGitHub e083471ec4 use argon2 with sync.Pool for better memory management (#11019) 2020-12-03 19:23:19 -08:00
HarshavardhanaandGitHub 80d31113e5 fix: etcd import paths again depend on v3.4.14 release (#11020)
Due to botched upstream renames of project repositories
and incomplete migration to go.mod support, our current
dependency version of `go.mod` had bugs i.e it was
using commits from master branch which didn't have
the required fixes present in release-3.4 branches

which leads to some rare bugs

https://github.com/etcd-io/etcd/pull/11477 provides
a workaround for now and we should migrate to this.

release-3.5 eventually claims to fix all of this
properly until then we cannot use /v3 import right now
2020-12-03 11:35:18 -08:00
HarshavardhanaandGitHub a6113b2315 update minio-go version v7.0.6 (#11012) 2020-12-01 19:15:38 -08:00
Klaus PostandGitHub f139a19238 Upgrade compress and pgzip package (#10992)
Should provide faster decompression for s3 select and other places where it is used.
2020-11-27 10:10:15 -08:00
Klaus PostandGitHub a75fafdbe2 Remove msgp workaround (#10964)
The error in `github.com/philhofer/fwd` was quickly fixed through 
https://github.com/philhofer/fwd/pull/22 - update the dependency 
and remove the workaround.
2020-11-24 11:58:10 -08:00
Shireesh AnjalandGitHub 14a7ae8586 Remove platform specific structure definitions (#10935)
Instead of having less/more fields inside a structure depending on the
platform (non-linux/linux), it would be better to have the same standard
definition in all platforms, and certain fields of the structure to be
populated or left unpopulated depending on the platform.
2020-11-21 09:41:33 -08:00
Poorna KrishnamoorthyandHarshavardhana 251c1ef6da Add support for replication of object tags, retention metadata (#10880) 2020-11-19 18:56:09 -08:00
Harshavardhana 9a34fd5c4a Revert "Revert "Add delete marker replication support (#10396)""
This reverts commit 267d7bf0a9.
2020-11-19 18:43:58 -08:00
HarshavardhanaandGitHub 7ff8128f15 update jstream to latest release v1.0.1 (#10909) 2020-11-17 09:13:28 -08:00
Harshavardhana 267d7bf0a9 Revert "Add delete marker replication support (#10396)"
This reverts commit 50c10a5087.

PR is moved to origin/dev branch
2020-11-12 11:43:14 -08:00
Poorna KrishnamoorthyandGitHub 50c10a5087 Add delete marker replication support (#10396)
Delete marker replication is implemented for V2
configuration specified in AWS spec (though AWS
allows it only in the V1 configuration).

This PR also brings in a MinIO only extension of
replicating permanent deletes, i.e. deletes specifying
version id are replicated to target cluster.
2020-11-10 15:24:14 -08:00
Klaus PostandGitHub 311ab43d4c Upgrade msgp to official release (#10858)
Code identical, but official release.
2020-11-09 07:52:24 -08:00
Klaus PostandGitHub 47d715f642 Update msgp to main branch (#10835)
The main branch has error unwrapping to clean up logs on canceled contexts, etc.

Asked for a new release: https://github.com/tinylib/msgp/issues/282
2020-11-05 08:13:03 -08:00
Klaus PostandGitHub 6b14c4ab1e Optimize decryptObjectInfo (#10726)
`decryptObjectInfo` is a significant bottleneck when listing objects.

Reduce the allocations for a significant speedup.

https://github.com/minio/sio/pull/40

```
λ benchcmp before.txt after.txt
benchmark                          old ns/op     new ns/op     delta
Benchmark_decryptObjectInfo-32     24260928      808656        -96.67%

benchmark                          old MB/s     new MB/s     speedup
Benchmark_decryptObjectInfo-32     0.04         1.24         31.00x

benchmark                          old allocs     new allocs     delta
Benchmark_decryptObjectInfo-32     75112          48996          -34.77%

benchmark                          old bytes     new bytes     delta
Benchmark_decryptObjectInfo-32     287694772     4228076       -98.53%
```
2020-10-29 09:34:20 -07:00
HarshavardhanaandGitHub 23773bb32b update NTP package for accurate time resolution fixes (#10670) 2020-10-14 12:29:20 +05:30
Poorna KrishnamoorthyandGitHub dbbed6f7f0 update minio-go dependency (#10634) 2020-10-06 08:37:09 -07:00
f1418a50f0 add NVMe drive info [model num, serial num, drive temp. etc.] (#10613)
* add NVMe drive info [model num, serial num, drive temp. etc.]
* Ignore fuse partitions
* Add the nvme logic only for linux
* Move smart/nvme structs to a separate file

Co-authored-by: wlan0 <sidharthamn@gmail.com>
2020-10-04 10:18:46 -07:00
HarshavardhanaandGitHub c6a9a94f94 fix: optimize ServerInfo() handler to avoid reading config (#10626)
fixes #10620
2020-10-02 16:19:44 -07:00
Anis ElleuchandGitHub d1ed1da8c6 build: check-gen should exit with some changes in generated code (#10590) 2020-10-02 11:10:39 -07:00
Anis ElleuchandGitHub e0c04a2da0 Makefile: Check for any non committed auto-generated code (#10579)
Always check if the auto-generated code is still compatible with the
existing written code to avoid a possible forgetting or sometimes a non
intentional change.
2020-09-28 13:33:34 -07:00
Praveen raj ManiandGitHub b880796aef Set the maximum open connections limit in PG and MySQL target configs (#10558)
As the bulk/recursive delete will require multiple connections to open at an instance,
The default open connections limit will be reached which results in the following error

```FATAL:  sorry, too many clients already```

By setting the open connections to a reasonable value - `2`, We ensure that the max open connections
will not be exhausted and lie under bounds.

The queries are simple inserts/updates/deletes which is operational and sufficient with the
the maximum open connection limit is 2.

Fixes #10553

Allow user configuration for MaxOpenConnections
2020-09-24 22:20:30 -07:00
HarshavardhanaandGitHub 02c1a08a5b fix: make sure to lock CopyObject for in-place updates (#10492) 2020-09-15 20:44:48 -07:00
Klaus PostandGitHub 16e1a25bc0 go.mod: Require Go 1.14 (#10441)
MinIO requires Go 1.14, so bump mod version.
2020-09-09 11:23:54 -07:00
Gaige B PaulsenandGitHub 859ef52886 update for smartos build (solaris too) (#10378) 2020-08-31 10:19:25 -07:00
Harshavardhana 7e80afdd7f remove nancy ignore list, as upstream issue is fixed 2020-08-26 15:45:42 -07:00
Praveen raj ManiandGitHub d0c910a6f3 Support https and basic-auth for elasticsearch notification target (#10332) 2020-08-23 09:43:48 -07:00
HarshavardhanaandGitHub 81c90ae430 move to gorilla mux 1.8.0 (#10328) 2020-08-22 23:14:02 -07:00
HarshavardhanaandGitHub 83a82d818e allow lock tolerance to match storage-class drive tolerance (#10270) 2020-08-14 18:17:14 -07:00
poornasandGitHub 79e21601b0 fix: web handlers to enforce replication (#10249)
This PR also preserves source ETag for replication
2020-08-12 17:32:24 -07:00
HarshavardhanaandGitHub 1e2ebc9945 feat: time to bring back http2.0 support (#10230)
Bonus move our CI/CD to go1.14
2020-08-10 09:02:29 -07:00
HarshavardhanaandGitHub cd04600862 update minio-go version to latest v7.0.3 (#10201)
- Add methods to set/remove replication rules (poornas)
- fix: only SSE-C headers should be applied to destination (Harshavardhana)
- fix: avoid data race by copying the buffer (Harshavardhana)
- remove deprecated build badges (Harshavardhana)
- fix: handle readFull bug with certain readers (Harshavardhana)
- fix a typo in README.md (Julien K)
- lifecycle: Fix marshaling expiration date/days (Anis Elleuch)
- add replication-status, expiration headers (Harshavardhana)
- Return object's version id in StatObject(Anis Elleuch)
- display appropriate funcName with nested callers (Harshavardhana)
- allow KMS tests to be run in the CI/CD (Harshavardhana)
- fix: removing lifecycle properly (Harshavardhana)
- feat: Add ListenNotification API to listen for all events (Harshavardhana)
2020-08-05 08:14:36 -07:00
Harshavardhana db6bba709a update CREDITS with minio/selfupdate project 2020-07-31 23:58:30 -07:00
poornasandGitHub b46ab7e921 Rename replication target handler (#10142)
Rename replication target handler to a generic bucket target handler
2020-07-28 11:50:47 -07:00
HarshavardhanaandGitHub 3a73f1ead5 refactor server update behavior (#10107) 2020-07-23 08:03:31 -07:00
poornasandGitHub c43da3005a Add support for server side bucket replication (#9882) 2020-07-21 17:49:56 -07:00
HarshavardhanaandGitHub 4bfc50411c fix: return versionId in tagging APIs (#10068) 2020-07-16 22:38:58 -07:00
Anis ElleuchandGitHub 778e9c864f Move dependency from minio-go v6 to v7 (#10042) 2020-07-14 09:38:05 -07:00
Andreas AuernhammerandGitHub 8d425e3372 fix etcd module dependency (#10032)
This commit fixes a dependency resolution problem w.r.t. minio => etcd.
Building any project depending on minio (e.g. mc) currently fails with
latest master since the module replace directive is not honored for
transitive / indirect dependencies.

This commit fixes this by adding the etcd module directly instead of
using a module replace instruction.
2020-07-13 07:46:34 -07:00
Harshavardhana 143f9371c6 fix: loading users regression
additionally also move to latest gorilla/mux master
to fix the DNS style bucket routing regression

resolves #10022
resolves #10023
2020-07-11 14:03:27 -07:00
HarshavardhanaandGitHub 72e0745e2f fix: migrate to go.etcd.io import path (#9987)
with the merge of https://github.com/etcd-io/etcd/pull/11823
etcd v3.5.0 will now have a properly imported versioned path

this fixes our pending migration to newer repo
2020-07-07 19:04:29 -07:00
HarshavardhanaandGitHub c087a05b43 fix: simplify data structure before release (#9968)
- additionally upgrade to msgp@v1.1.2
- change StatModTime,StatSize fields as
  simple Size/ModTime
- reduce 50000 entries per List batch to 10000
  as client needs to wait too long to see the
  first batch some times which is not desired
  and it is worth we write the data as soon
  as we have it.
2020-07-04 12:25:53 -07:00
HarshavardhanaandGitHub 3b813148b3 update gorilla deps for query parsing performance improvements (#9929) 2020-06-28 11:37:56 -07:00
Klaus PostandGitHub 2d0f65a5e3 Add archived parquet as int. package (#9912)
Since github.com/minio/parquet-go is archived add it as internal package.
2020-06-25 07:31:16 -07:00