github/minio
4
0
Fork 0
mirror of https://github.com/minio/minio.git synced 2024-04-21 12:41:46 +00:00
Code Issues Releases Wiki Activity
4,282 Commits 6 Branches 471 Tags
8085ba4494de3ca5dbd5303053fecc13f3febf05
Commit Graph
70 Commits
Author SHA1 Message Date
Aditya ManthramurthyandDee Koder a02575ebf9 Bump up minio-go to (fixes #4243) (#4256) 2017-05-04 13:43:23 -07:00
Krishna SrinivasandHarshavardhana e5b2e25caf gateway-s3: vendor-update minio-go (#4220) 2017-05-02 18:46:39 -07:00
Remco VerhoefandHarshavardhana 3a539ce660 Implement gateway S3 support (#3940) 2017-04-27 11:26:00 -07:00
HarshavardhanaandGitHub f1d7780167 lock: Vendorize all the new changes made in minio/dsync (#4154) 
Fixes #4139
 2017-04-19 14:22:35 -07:00
Bala FAandHarshavardhana de204a0a52 Add extensive endpoints validation (#4019) 2017-04-11 15:44:27 -07:00
Krishna SrinivasandHarshavardhana cea4cfa3a8 Implement S3 Gateway to third party cloud storage providers. (#3756) 
Currently supported backend is Azure Blob Storage.

```
export MINIO_ACCESS_KEY=azureaccountname
export MINIO_SECRET_KEY=azureaccountkey
minio gateway azure
```
 2017-03-16 12:21:58 -07:00
HarshavardhanaandGitHub 3edff1501e Vendor upstream redis library instead of our fork. (#3913) 
We forked the upstream to address a build issue on
go 1.6 - that is long done and we don't need
to manage our forks anymore.
 2017-03-16 08:22:47 -07:00
HarshavardhanaandGitHub 9eb8e375c5 cli: Make sure to add --help flag for subcommands. (#3773) 
--help is now back and prints properly with command
help template.
 2017-02-19 20:46:06 -08:00
Harshavardhana 25eeb88a8f cli: Bring upstream changes to minio server. 2017-02-15 17:30:31 -08:00
Andreas AuernhammerandHarshavardhana f38222c0cc update the blake2b implementation (#3724) 
Fixes a performance bug caused by SSE-AVX register savings on amd64.
 2017-02-09 15:01:00 -08:00
Harshavardhana d41dcb784b Move to blake2b-simd due to perf problems in golang.org/x/crypto 
Ref https://github.com/golang/go/issues/18563
 2017-01-24 18:07:34 -08:00
Krishna SrinivasandHarshavardhana cead24b0f7 miniobrowser: Bring Minio browser source into minio repo. (#3617) 2017-01-23 18:07:22 -08:00
Krishna SrinivasandHarshavardhana 4e926b292f vendor-update: Minio Browser (#3609) 2017-01-22 21:02:09 -08:00
Bala.FA 6d10f4c19a Adopt dsync interface changes and major cleanup on RPC server/client. 
* Rename GenericArgs to AuthRPCArgs
* Rename GenericReply to AuthRPCReply
* Remove authConfig.loginMethod and add authConfig.ServiceName
* Rename loginServer to AuthRPCServer
* Rename RPCLoginArgs to LoginRPCArgs
* Rename RPCLoginReply to LoginRPCReply
* Version and RequestTime are added to LoginRPCArgs and verified by
  server side, not client side.
* Fix data race in lockMaintainence loop.
 2017-01-02 20:57:42 +05:30
Andreas AuernhammerandHarshavardhana 1ac36a95aa replace blake2b implementation (#3481) 
* replace blake2b implementation
replace the blake2b-simd with the golang/x/crypto implementation

```
name        old time/op    new time/op     delta
Size64-8       715ns ±13%      614ns ± 3%    ~     (p=0.084 n=6+6)
Size128-8      612ns ± 5%      634ns ± 8%    ~     (p=0.084 n=6+6)
Size1K-8      2.18µs ± 5%     2.09µs ± 7%    ~     (p=0.084 n=6+6)
Size8K-8      13.1µs ± 2%     13.4µs ± 3%    ~     (p=0.084 n=6+6)
Size32K-8     48.5µs ± 1%     49.5µs ± 3%    ~     (p=0.775 n=6+6)
Size128K-8     199µs ± 0%      198µs ± 3%    ~     (p=0.468 n=6+6)

name        old speed      new speed       delta
Size64-8    92.6MB/s ±11%  104.2MB/s ± 3%    ~     (p=0.139 n=6+6)
Size128-8    208MB/s ± 6%    202MB/s ± 8%    ~     (p=0.102 n=6+6)
Size1K-8     466MB/s ± 7%    492MB/s ± 7%    ~     (p=0.139 n=6+6)
Size8K-8     621MB/s ± 2%    610MB/s ± 3%    ~     (p=0.102 n=6+6)
Size32K-8    672MB/s ± 2%    669MB/s ± 1%    ~     (p=0.818 n=6+6)
Size128K-8   657MB/s ± 1%    672MB/s ± 0%  +2.28%  (p=0.002 n=6+6)

name        old time/op   new time/op   delta
Size64-4      334ns ± 1%    243ns ± 0%  -27.14%  (p=0.029 n=4+4)
Size128-4     296ns ± 1%    242ns ± 0%  -18.21%  (p=0.029 n=4+4)
Size1K-4     1.44µs ± 0%   1.28µs ± 0%  -10.83%  (p=0.029 n=4+4)
Size8K-4     10.0µs ± 0%    9.4µs ± 0%   -6.23%  (p=0.029 n=4+4)
Size32K-4    39.8µs ± 1%   37.3µs ± 0%   -6.31%  (p=0.029 n=4+4)
Size128K-4    162µs ± 3%    149µs ± 0%   -7.72%  (p=0.029 n=4+4)

name        old speed     new speed     delta
Size64-4    192MB/s ± 1%  263MB/s ± 0%  +37.24%  (p=0.029 n=4+4)
Size128-4   431MB/s ± 0%  526MB/s ± 0%  +22.04%  (p=0.029 n=4+4)
Size1K-4    713MB/s ± 0%  800MB/s ± 0%  +12.17%  (p=0.029 n=4+4)
Size8K-4    815MB/s ± 0%  869MB/s ± 0%   +6.64%  (p=0.029 n=4+4)
Size32K-4   823MB/s ± 1%  878MB/s ± 0%   +6.72%  (p=0.029 n=4+4)
Size128K-4  810MB/s ± 3%  877MB/s ± 0%   +8.23%  (p=0.029 n=4+4)
```
See: https://go-review.googlesource.com/#/c/34319/
 2016-12-21 14:20:01 -08:00
Harshavardhana 1d134c1a94 admin: Fix wrong vendorized location for s3signer and s3utils. 2016-12-20 15:18:37 -08:00
HarshavardhanaandGitHub faa6b1e925 vendorize deps for snappy, blake2b and sha256 (#3476) 
Bring in new optimization and portability changes.

Fixes https://github.com/minio/minio-go/issues/578
 2016-12-19 19:32:55 -08:00
Krishnan ParthasarathiandHarshavardhana a609a4126c Vendorize recent fixes in dsync (#3365) 
* Update 'i' only if lock grant read from buffered channel
 2016-11-28 23:03:46 -08:00
HarshavardhanaandGitHub 12c1abed98 Vendorize with bug fixes from minio browser. (#3341) 
This patch brings in changes from miniobrowser repo.

- Bucket policy UI and functionality fixes by @krishnasrinivas
- Bucket policy implementation by @balamurugana
- UI changes and new functionality changing password etc. @rushenn
- UI and new functionality for sharing URLs, deleting files
  @rushenn and @krishnasrinivas.
- Other misc fixes by @vadmeste @brendanashworth
 2016-11-23 17:31:11 -08:00
Harshavardhana 7a5bbf7a2e Revert "Vendor update for dsync, fixing major go routine leak issue. (#3308)" 
This reverts commit 273228fafa.
 2016-11-23 15:17:41 -08:00
Karthic RaoandHarshavardhana 273228fafa Vendor update for dsync, fixing major go routine leak issue. (#3308) 2016-11-21 10:47:48 -08:00
FrankandHarshavardhana ea406754a6 New dsync and added ForceUnlock to lock rpc server (#2956) 
* Update dsync and added ForceUnlock function
* Added test cases for ForceUnlock
 2016-10-17 01:53:29 -07:00
HarshavardhanaandGitHub a681af6953 Update minio browser with new changes. (#2940) 
- Bucket policy set/unset support.
- Shareable URL support.
- Delete object support.
 2016-10-15 08:51:53 -07:00
FrankandHarshavardhana e53a9f6cab Update vendorized version of dsync that relaxes read quorum to N/2 (#2874) 2016-10-07 08:02:59 -07:00
Bala FAandHarshavardhana aa579bbc20 web: add method to get all policies for given bucket name. (#2756) 
Refer #1858
 2016-09-22 23:06:45 -07:00
FrankandHarshavardhana df2ef64d20 Upgrade to new dsync version incl. stale lock detection (#2708) 2016-09-16 00:30:55 -07:00
Harshavardhana f655592ff5 vendorize new dsync with new changes of logging. 2016-09-13 21:18:30 -07:00
Harshavardhana 9605fde04d controller/auth: Implement JWT based authorization for controller. (#2544) 
Fixes #2474
 2016-09-13 21:18:30 -07:00
Krishnan ParthasarathiandHarshavardhana bda6bcd5be Layered rpc-client implementation (#2512) 2016-09-13 21:18:30 -07:00
Krishnan ParthasarathiandHarshavardhana c33d1b8ee6 Vendorize minio/dsync for server-side read lock (#2484) 
- Prevention of stale lock accumulation.
- Removal of dead code.
 2016-09-13 21:18:30 -07:00
Harshavardhana 83074ed57e Add missing rpc-client.go - missing in previous commit. 2016-09-13 21:18:30 -07:00
Krishnan ParthasarathiandHarshavardhana 64a7e6992e Vendorize rpc reconnect changes from minio/dsync (#2405) 2016-09-13 21:18:30 -07:00
Krishnan ParthasarathiandHarshavardhana b0f3f94163 unify single-node and distributed namespace locking (#2401) 2016-09-13 21:18:30 -07:00
Krishnan ParthasarathiandHarshavardhana b7c169d71d object layer initialization using distributed locking (#2397) 
* vendorized latest minio/dsync

* wip - object layer initialization using distributed locking
 2016-09-13 21:18:30 -07:00
Krishnan ParthasarathiandHarshavardhana e55926e8cf distribute: Make server work with multiple remote disks 
This change initializes rpc servers associated with disks that are
local. It makes object layer initialization on demand, namely on the
first request to the object layer.

Also adds lock RPC service vendorized minio/dsync
 2016-09-13 21:18:30 -07:00
Bala FAandHarshavardhana 7431acb2c4 web: add handlers for set/get bucket policy. (#2486) 
Refer #1858
 2016-08-30 10:04:50 -07:00
HarshavardhanaandGitHub 975eb31973 api/bucketPolicy: Use minio-go/pkg/set and fix bucket policy regression. (#2506) 
Current master has a regression 'mc policy <policy-type> alias/bucket/prefix'
does not work anymore, due to the way new minio-go changes do json marshalling.
This led to a regression on server side when a ``prefix`` is provided
policy is rejected as malformed from th server which is not the case with
AWS S3.

This patch uses the new ``minio-go/pkg/set`` package to address the
unmarshalling problems.

Fixes #2503
 2016-08-20 03:16:38 -07:00
HarshavardhanaandGitHub 0a3d43273f vendor: sha256 32bit updated. (#2459) 2016-08-16 16:19:29 -07:00
FrankandHarshavardhana f239fcac67 Switched to faster minio/sha256-simd implementation (#2320) 2016-07-28 14:44:37 -07:00
HarshavardhanaandAnand Babu (AB) Periasamy f248089523 api: Implement bucket notification. (#2271) 
* Implement basic S3 notifications through queues

Supports multiple queues and three basic queue types:

1. NilQueue -- messages don't get sent anywhere
2. LogQueue -- messages get logged
3. AmqpQueue -- messages are sent to an AMQP queue

* api: Implement bucket notification.

Supports two different queue types

- AMQP
- ElasticSearch.

* Add support for redis
 2016-07-23 22:51:12 -07:00
HarshavardhanaandAnand Babu (AB) Periasamy 169c72cdab vendor: Bring new updates from blake2b-simd repo. (#2094) 
This vendorization is needed to bring in new improvements
and support for AVX2 and SSE.

Fixes #2081
 2016-07-06 18:24:31 -07:00
HarshavardhanaandAnand Babu (AB) Periasamy 48ac34919f browser: Add new release for ui-assets.go (#2070) 
update `ui-assets.go` using `x-amz-date` for JSON rpc.
 2016-07-02 10:54:17 -07:00
HarshavardhanaandAnand Babu (AB) Periasamy d484157d67 XL/bitrot: Migrate to new blake2b-simd SIMD optimized implementation. (#2031) 
Thanks for Frank Wessels <fwessels@xs4all.nl> for all the heavy lifting work.

Comparative benchmarks are as below.
```
benchmark                old ns/op     new ns/op     delta
BenchmarkHash64-4        742           411           -44.61%
BenchmarkHash128-4       681           346           -49.19%
BenchmarkWrite1K-4       4239          1497          -64.69%
BenchmarkWrite8K-4       33633         11514         -65.77%
BenchmarkWrite32K-4      134091        45947         -65.73%
BenchmarkWrite128K-4     537976        183643        -65.86%

benchmark                old MB/s     new MB/s     speedup
BenchmarkHash64-4        86.18        155.51       1.80x
BenchmarkHash128-4       187.96       369.10       1.96x
BenchmarkWrite1K-4       241.55       683.87       2.83x
BenchmarkWrite8K-4       3897.06      11383.41     2.92x
BenchmarkWrite32K-4      977.48       2852.63      2.92x
BenchmarkWrite128K-4     243.64       713.73       2.93x
```

Fixes #2030
 2016-06-29 02:06:35 -07:00
Harshavardhana 2e4ab71303 Web: Update with ui changes. (#1808) 2016-05-31 02:01:02 -07:00
Harshavardhana 444d1f8a65 miniobrowser: Vendorize to new changes in miniobrowser. 2016-04-21 20:35:48 -07:00
Harshavardhana 8603185f2f browser: Add new ui-assets.go 2016-04-08 01:47:30 -07:00
Harshavardhana ba3a5805c1 vendorize: Add updated ui-assets.go. 2016-04-02 17:27:36 -07:00
Krishna Srinivas 331890c4c8 UI-handler: remove minio-go dependancy. 2016-04-01 13:56:32 +05:30
Harshavardhana aaf97ea02c config/main: Re-write config files - add to new config v3 
- New config format.

```
{
	"version": "3",
	"address": ":9000",
    "backend": {
          "type": "fs",
          "disk": "/path"
    },
	"credential": {
		"accessKey": "WLGDGYAQYIGI833EV05A",
		"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
	},
	"region": "us-east-1",
	"logger": {
		"file": {
			"enable": false,
			"fileName": "",
			"level": "error"
		},
		"syslog": {
			"enable": false,
			"address": "",
			"level": "debug"
		},
		"console": {
			"enable": true,
			"level": "fatal"
		}
	}
}
```

New command lines in lieu of supporting XL.

Minio initialize filesystem backend.
~~~
$ minio init fs <path>
~~~

Minio initialize XL backend.
~~~
$ minio init xl <url1>...<url16>
~~~

For 'fs' backend it starts the server.
~~~
$ minio server
~~~

For 'xl' backend it waits for servers to join.
~~~
$ minio server
... [PROGRESS BAR] of servers connecting
~~~

Now on other servers execute 'join' and they connect.
~~~
....
minio join <url1> -- from <url2> && minio server
minio join <url1> -- from <url3> && minio server
...
...
minio join <url1> -- from <url16> && minio server
~~~
 2016-03-23 19:16:09 -07:00
Harshavardhana 5606232567 vendor: Update ui-assets with new changes and release. 2016-03-10 17:36:32 -08:00