Commit Graph
382 Commits
Author SHA1 Message Date
Anis ElleuchandHarshavardhana e2216a0936 exponentialBackoffWait returns zero after some retries, limit attempt number inside (#3245) 2016-11-11 09:49:30 -08:00
HarshavardhanaandGitHub e8d9d710d0 rpc: Protect racy access of internal auth states. (#3238)
Fixes #3232
2016-11-11 00:14:32 -08:00
HarshavardhanaandGitHub a8ab02a73a v4/presign: Fix presign requests when there are more signed headers. (#3222)
This fix removes a wrong logic which fails for requests which
have more signed headers in a presign request.

Fixes #3217
2016-11-10 21:57:15 -08:00
Aditya ManthramurthyandHarshavardhana cf022de4d5 Add tests for s3PeerAPIHandlers (Fixes #3067) (#3242) 2016-11-10 16:43:04 -08:00
Bala FAandHarshavardhana 3995e21c5b fix: Ignore object not found error in RemoveObject() in web-handler. (#3228)
Fixes #3181
2016-11-10 15:02:03 -08:00
HarshavardhanaandGitHub 2f7fb78692 rpc: Our rpcClient should make an attempt to reconnect. (#3221)
rpcClient should attempt a reconnect if the call fails
with 'rpc.ErrShutdown' this is needed since at times when
the servers are taken down and brought back up.

The hijacked connection from net.Dial is usually closed.

So upon first attempt rpcClient might falsely indicate that
disk to be down, to avoid this state make another dial attempt
to really fail.

Fixes #3206
Fixes #3205
2016-11-10 07:44:41 -08:00
Bala FAandHarshavardhana cf2fb30ac7 event: Add event notification for object deletion in web browser. (#3226)
Fixes #3201
2016-11-10 07:42:55 -08:00
HarshavardhanaandGitHub 51d1e6f75b Fix missing CompleteMultipartUpload Etag. (#3227)
Fixes #3224
2016-11-10 07:41:02 -08:00
HarshavardhanaandGitHub 2f373684f5 Fix the server startup messages and help text. (#3211) 2016-11-09 23:37:12 -08:00
Aditya ManthramurthyandHarshavardhana dd0698d14c Improve namespace lock API: (#3203)
- abstract out instrumentation information.
- use separate lockInstance type that encapsulates the nsMutex, volume,
  path and opsID as the frontend or top-level lock object.
2016-11-09 10:58:41 -08:00
HarshavardhanaandGitHub 3e67bfcc88 heal: Print heal command appropriately without export path. (#3208)
Fixes #3204
2016-11-09 10:50:14 -08:00
Anis ElleuchandHarshavardhana ea579f5b69 Avoid shutdown fs multiple times and create a new fs in each loop (#3213) 2016-11-09 10:10:14 -08:00
Aditya ManthramurthyandHarshavardhana d44e9d6da9 Prevent weird messages from rpc lib on start (Fixes #3209): (#3212)
This is done by not making the methods of the BucketMetaState interface
as methods (via type nesting) on the type implementing
RPCs (s3PeerAPIHandlers).
2016-11-08 23:47:44 -08:00
Bala FAandHarshavardhana 9c2cfb5cb6 tests: Add missing unit test cases for AbortMultipartUploadHandler(). (#3200)
Fixes #3070
2016-11-08 16:25:00 -08:00
Bala FAandHarshavardhana 91a0ade908 tests: add unit test for HeadObjectHandler (#3197)
Fixes #3068
2016-11-07 16:02:27 -08:00
Aditya ManthramurthyandHarshavardhana 85a5c358d8 Add bucket metadata state client/handler (Fixes #3022) (#3152)
- Adds an interface to update in-memory bucket metadata state called
  BucketMetaState - this interface has functions to:
     - update bucket notification configuration,
     - bucket listener configuration,
     - bucket policy configuration, and
     - send bucket event

- This interface is implemented by `localBMS` a type for manipulating
  local node in-memory bucket metadata, and by `remoteBMS` a type for
  manipulating remote node in-memory bucket metadata.

- The remote node interface, makes an RPC call, but the local node
  interface does not - it updates in-memory bucket state directly.

- Rename mkPeersFromEndpoints to makeS3Peers and refactored it.

- Use arrayslice instead of map in s3Peers struct

- `s3Peers.SendUpdate` now receives an arrayslice of peer indexes to
  send the request to, with a special nil value slice indicating that
  all peers should be sent the update.

- `s3Peers.SendUpdate` now returns an arrayslice of errors, representing
  errors from peers when sending an update. The array positions
  correspond to peer array s3Peers.peers

Improve globalS3Peers:

- Make isDistXL a global `globalIsDistXL` and remove from s3Peers

- Make globalS3Peers an array of (address, bucket-meta-state) pairs.

- Fix code and tests.
2016-11-07 12:09:24 -08:00
HarshavardhanaandGitHub 33c771bb3e tests: Add tests for browser peer rpc. Fixes #3062 (#3189) 2016-11-07 11:43:35 -08:00
Karthic RaoandHarshavardhana 286a8924fd Add leak detection to object-handler tests. (#3195) 2016-11-06 21:53:50 -08:00
Karthic RaoandHarshavardhana efca29b00e Fix typos and comments in leak_detect_test.go. (#3193) 2016-11-06 21:02:29 -08:00
HarshavardhanaandGitHub 9161016962 tests: Improve coverage on signature v4 tests. (#3188)
Fixes #3065
2016-11-06 11:47:16 -08:00
Anis ElleuchandHarshavardhana 5ff30777e1 Rewrite connection muxer peek process to avoid server blocking by silent clients (#3187) 2016-11-06 11:41:01 -08:00
Anis ElleuchandHarshavardhana 754c0770d6 Merge ListenAndServe and ListenAndServeTLS for simplification purpose (#3186) 2016-11-05 20:32:13 -07:00
HarshavardhanaandGitHub 1105508453 connection muxer should use bufio.Reader to be simpler. (#3177) 2016-11-05 12:57:31 -07:00
HarshavardhanaandGitHub 1ba497950c Fix net.Listener to fully close the underlying socket. (#3171)
Leads to races and accepting connections. This patch implements
a way to reject accepting new connections.
2016-11-05 09:43:28 -07:00
Aditya ManthramurthyandHarshavardhana eb1bc67db1 Refactor: simplify signV4TrimAll() (#3179) 2016-11-04 13:52:22 -07:00
Krishna SrinivasandHarshavardhana 8408dfaa6c bootup-validation: Allow port configuration only using --address option. (#3166) 2016-11-04 12:14:19 -07:00
HarshavardhanaandGitHub d192044915 router: PathPrefix router was wrong. (#3172) 2016-11-04 12:13:22 -07:00
Anis ElleuchandHarshavardhana d9bab6b3bd sigv4: Trim and shrink spaces in headers values (#3162) 2016-11-03 16:41:25 -07:00
Anis ElleuchandHarshavardhana e6965ca066 Quit initializing disks process when term signal is invoked (#3163) 2016-11-02 15:27:36 -07:00
HarshavardhanaandGitHub d9674f7524 Improve coverage of web-handlers.go (#3157)
This patch additionally relaxes the requirement for
accesskeys to be in a regexy set of values.

Fixes #3063
2016-11-02 14:45:11 -07:00
Anis ElleuchandHarshavardhana 79601d27f2 Use endpoint url when printing disks status in distributed mode (#3151) 2016-11-02 08:51:06 -07:00
Aditya ManthramurthyandHarshavardhana 226a69fe15 Add test for updateUploadJSON (Fixes #3060) (#3155) 2016-11-01 20:04:32 -07:00
HarshavardhanaandGitHub 9bb799462e Start all listeners when a given host resolves to multiple IPs. (#3145)
Default golang net.Listen only listens on the first IP when
host resolves to multiple IPs.

This change addresses a problem for example your ``/etc/hosts``
has entries as following

```
127.0.1.1  minio1

192.168.1.10 minio1
```

Trying to start minio as

```
minio server --address "minio1:9001" ~/Photos
```

Causes the minio server to be bound only to "127.0.1.1" which
is an incorrect behavior since we are generally interested in
`192.168.1.10` as well.

This patch addresses this issue if the hostname is resolvable
and gives back list of addresses associated with that hostname
we just bind on all of them as it is the expected behavior.
2016-11-01 15:38:28 -07:00
Karthic RaoandHarshavardhana 8bffa78f7f Fix Instance type during benchmarks. (#3147)
- The benchmark initialization function was not taking into account the
  instance type (FS/XL), was using XL ObjectLayer even for FS
  benchmarks.
- This was leading to incorrect benchmark results for FS related
  benchmarks.
- The fix takes into account the instance type (FS/XL) and correctly
  returns FS backend for FS benchmarks.
2016-11-01 10:21:16 -07:00
Anis ElleuchandHarshavardhana 4b302173ae fs: Remove object append file if found since it is not always used (#3139) 2016-11-01 00:56:03 -07:00
Karthic RaoandHarshavardhana 9db2205db3 Fix List-Object Benchmark failure. (#3142) 2016-10-31 13:57:30 -07:00
Aditya ManthramurthyandHarshavardhana 6914fe436c Remove unused function. (#3143) 2016-10-31 12:18:15 -07:00
Anis ElleuchandHarshavardhana 807cc3c28d Tracing locking errors for better debugging (#3140) 2016-10-31 10:17:14 -07:00
HarshavardhanaandGitHub f3c6c55719 posix: Fix windows performance issues. (#3132)
Do not attempt to fetch volume/drive information for
each i/o situation. In our case we do this in all calls
`posix.go` this in-turn created a terrible situation for
windows. This issue does not affect the i/o path on Unix
platforms since statvfs calls are in the range of micro
seconds on these platforms.

This verification is only needed during startup and we
let things fail at a later stage on windows.
2016-10-31 09:34:44 -07:00
Karthic RaoandHarshavardhana 3e8cb8c937 build: ineffassign fixes. (#3134) 2016-10-30 23:32:46 -07:00
Aditya ManthramurthyandHarshavardhana dd6ecf1193 Read/write quorum algo with uploads.json in xl (Fixes #3123) (#3124)
- Reads and writes of uploads.json in XL now uses quorum for
  newMultipart, completeMultipart and abortMultipart operations.

- Each disk's `uploads.json` file is read and updated independently for
  adding or removing an upload id from the file. Quorum is used to
  decide if the high-level operation actually succeeded.

- Refactor FS code to simplify the flow, and fix a bug while reading
  uploads.json.
2016-10-30 09:27:29 -07:00
Anis ElleuchandHarshavardhana a47ce7ab22 Add support of fallocate for FS and XL backends (#3032) 2016-10-29 12:44:44 -07:00
Krishna SrinivasandHarshavardhana 0b3282ac9f Logging: errorIf fatalIf print in the format [file.go:82:funcName()] (#3127) 2016-10-29 02:34:16 -07:00
Krishna SrinivasandHarshavardhana 79b98b5c25 boot: getPath() should take care of simple directory exports. (#3122)
fixes #3120
2016-10-28 13:08:56 -07:00
Krishnan ParthasarathiandHarshavardhana 6a57f2c1f0 XL: Add more information to panic msg (#3119) 2016-10-28 08:46:03 -07:00
Krishna SrinivasandHarshavardhana 0fa2477cb0 boot: check parameter syntax before initializing the system. (#3114)
fixes #3112
2016-10-28 08:45:32 -07:00
Anis ElleuchandHarshavardhana 7ffb337cd7 Update control cmd USAGE to indicate that it is possible to put access secret keys in URL (#3115) 2016-10-27 13:26:11 -07:00
HarshavardhanaandGitHub 9e2d0ac50b Move to URL based syntax formatting. (#3092)
For command line arguments we are currently following

- <node-1>:/path ... <node-n>:/path

This patch changes this to

- http://<node-1>/path ... http://<node-n>/path
2016-10-27 03:30:52 -07:00
Aditya ManthramurthyandHarshavardhana 30dc11a931 No listener.json for single-node mode (Fixes #3052) (#3108)
In FS or single-node XL mode, there is no need to save listener
configuration to persistent storage. As there is only one server, if it
is restarted, any connected listenBucketAPI clients were disconnected
and will have to reconnect - so there is nothing to actually store.

This incidentally solves #3052 by avoiding the problem.
2016-10-26 20:13:00 -07:00
Anis ElleuchandHarshavardhana a15dc5fed5 Print message when creating the config file (#3089) 2016-10-26 18:44:22 -07:00