Commit Graph
235 Commits
Author SHA1 Message Date
jiukerandGitHub b49b39e99d fix: errNoSuchPolicy should use errors.Is (#16656) 2023-02-25 00:01:37 -08:00
Anis ElleuchandGitHub 94c6cb1323 tests: Add test for S3 API error codes (#16705) 2023-02-25 08:06:29 +05:30
Allan Roger ReidandGitHub 8bfe972bab Set meaningful message from minio with env variable KMS_SECRET_KEY (#16584) 2023-02-22 07:13:01 +05:30
PoornaandGitHub c33a237067 fix: under site replication disallow remote target modification (#16628) 2023-02-15 20:22:13 -08:00
HarshavardhanaandGitHub 37134e42d4 ignore io.EOF, io.ErrUnexpectedEOF on xl.meta reads in WalkDir() (#16625) 2023-02-15 07:12:48 -08:00
HarshavardhanaandGitHub e64b9f6751 fix: disallow SSE-C encrypted objects on replicated buckets (#16467) 2023-01-24 15:46:33 -08:00
PoornaandGitHub 6423e4c767 Remove site replication config if it succeeded locally (#16279) 2022-12-22 01:31:20 -08:00
709eb283d9 Add endpoints for managing IAM policies (#15897)
Co-authored-by: Taran <taran@minio.io>
Co-authored-by: ¨taran-p¨ <¨taran@minio.io¨>
Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2022-12-13 12:13:23 -08:00
Aditya ManthramurthyandGitHub e06127566d Add IAM API to attach/detach policies for LDAP (#16182) 2022-12-09 13:08:33 -08:00
HarshavardhanaandGitHub b03dd1af17 remove hard limit for number of buckets (#16194) 2022-12-08 12:24:03 -08:00
Aditya ManthramurthyandGitHub a30cfdd88f Bump up madmin-go to v2 (#16162) 2022-12-06 13:46:50 -08:00
Anis ElleuchandGitHub 44735be38e s3: Return correct error when Version is invalid in policy document (#16178) 2022-12-06 08:07:24 -08:00
Klaus PostandGitHub a713aee3d5 Run staticcheck on CI (#16170) 2022-12-05 11:18:50 -08:00
Aditya ManthramurthyandGitHub 4fb47cd568 fix: update admin IDP APIs to be more RESTful (#15896) 2022-10-31 14:52:26 -07:00
Krishnan ParthasarathiandGitHub 4523da6543 feat: introduce pool-level rebalance (#15483) 2022-10-25 12:36:57 -07:00
HarshavardhanaandGitHub 23b329b9df remove gateway completely (#15929) 2022-10-24 17:44:15 -07:00
Aditya ManthramurthyandGitHub 2d16e74f38 Add LDAP IDP Configuration APIs (#15840) 2022-10-19 11:00:10 -07:00
HarshavardhanaandGitHub c68910005b validate bucket before attempting batch replication (#15861) 2022-10-15 11:58:31 -07:00
Harshavardhana 2a13cc28f2 feat: implement support batch replication (#15554) 2022-10-05 23:00:43 -07:00
HarshavardhanaandGitHub b579163802 limit number of buckets to 500k (#15668)
500k is a reasonable limit for any single MinIO
cluster deployment, in future we may increase this
value.

However for now we are going to keep this limit.
2022-09-09 03:06:34 -07:00
Klaus PostandGitHub a9f1ad7924 Add extended checksum support (#15433) 2022-08-29 16:57:16 -07:00
PoornaandGitHub 21fe14201f replication: centralize healthcheck for remote targets (#15516)
This PR moves health check from minio-go client to being
managed on the server.

Additionally integrating health check into site replication
2022-08-16 17:46:22 -07:00
HarshavardhanaandGitHub 48640b1de2 fix: trim arn:aws:kms from incoming SSE aws-kms-key-id (#15540) 2022-08-16 11:28:30 -07:00
ebozdumanandGitHub b57e7321e7 Replaces 'disk'=>'drive' visible to end user (#15464) 2022-08-04 16:10:08 -07:00
Aditya ManthramurthyandGitHub af9bc7ea7d Add external IDP management Admin API for OpenID (#15152) 2022-07-05 18:18:04 -07:00
HarshavardhanaandGitHub 2bb6a3f4d0 cleanup site replication error handling (#15113)
site replication errors were printed at
various random locations, repeatedly - this
PR attempts to remove double logging and
capture all of them at a common place.

This PR also enhances the code to show
partial success and errors as well.
2022-06-20 10:48:11 -07:00
HarshavardhanaandGitHub 48e367ff7d reject resync start on misconfigured replication rules (#15041)
we expect resync to start on buckets with replication
rule ExistingObjects enabled, if not we reject such
calls.
2022-06-06 02:54:39 -07:00
HarshavardhanaandGitHub 60d0611ac2 use BadRequest HTTP status instead of Conflict for certain errors (#14900)
PutBucketVersioning API should return BadRequest for errors
instead of Conflict, Conflict is used for "AlreadyExists"
resource situations.
2022-05-11 13:44:16 -07:00
PoornaandGitHub 523670ba0d fix: site removal API error handling (#14870)
when the site is being removed is missing replication config. This can
happen when a new deployment is brought in place of a site that
is lost/destroyed and needs to delink old deployment from site
replication.
2022-05-06 12:40:34 -07:00
HarshavardhanaandGitHub 16bc11e72e fix: disallow newer policies, users & groups with space characters (#14845)
space characters at the beginning or at the end can lead to
confusion under various UI elements in differentiating the
actual name of "policy, user or group" - to avoid this behavior
this PR onwards we shall reject such inputs for newer entries.

existing saved entries will behave as is and are going to be
operable until they are removed/renamed to something more
meaningful.
2022-05-02 09:27:35 -07:00
PoornaandGitHub 93af4a4864 Handle non existent kms key correctly (#14329)
- in PutBucketEncryption API
- admin APIs for  `mc admin KMS key [create|info]`
- PutObject API when invalid KMS key is specified
2022-02-17 11:36:14 -08:00
HarshavardhanaandGitHub f527c708f2 run gofumpt cleanup across code-base (#14015) 2022-01-02 09:15:06 -08:00
HarshavardhanaandGitHub 1cf726348f return meaningful error for disabled users (#13968)
fixes #13958
2021-12-22 11:40:21 -08:00
Poorna KandGitHub b42cfcea60 Disallow versioning/replication change in cluster replication setup (#13910) 2021-12-15 10:37:08 -08:00
Aditya ManthramurthyandGitHub 4ce6d35e30 Add new site config sub-system intended to replace region (#13672)
- New sub-system has "region" and "name" fields.

- `region` subsystem is marked as deprecated, however still works, unless the
new region parameter under `site` is set - in this case, the region subsystem is
ignored. `region` subsystem is hidden from top-level help (i.e. from `mc admin
config set myminio`), but appears when specifically requested (i.e. with `mc
admin config set myminio region`).

- MINIO_REGION, MINIO_REGION_NAME are supported as legacy environment variables for server region.

- Adds MINIO_SITE_REGION as the current environment variable to configure the
server region and MINIO_SITE_NAME for the site name.
2021-11-25 13:06:25 -08:00
Krishnan ParthasarathiandGitHub 3da9ee15d3 Add MaxNoncurrentVersions to NoncurrentExpiration action (#13580)
This unit allows users to limit the maximum number of noncurrent 
versions of an object.

To enable this rule you need the following *ilm.json*
```
cat >> ilm.json <<EOF
{
    "Rules": [
        {
            "ID": "test-max-noncurrent",
            "Status": "Enabled",
            "Filter": {
                "Prefix": "user-uploads/"
            },
            "NoncurrentVersionExpiration": {
                "MaxNoncurrentVersions": 5
            }
        }
    ]
}
EOF
mc ilm import myminio/mybucket < ilm.json
```
2021-11-19 17:54:10 -08:00
661b263e77 add gocritic/ruleguard checks back again, cleanup code. (#13665)
- remove some duplicated code
- reported a bug, separately fixed in #13664
- using strings.ReplaceAll() when needed
- using filepath.ToSlash() use when needed
- remove all non-Go style comments from the codebase

Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2021-11-16 09:28:29 -08:00
HarshavardhanaandGitHub 68c5ad83fb fix: backend not reachable should be more descriptive (#13634) 2021-11-10 22:33:17 -08:00
HarshavardhanaandGitHub 4ed0eb7012 remove double reads updating object metadata (#13542)
Removes RLock/RUnlock for updating metadata,
since we already take a write lock to update
metadata, this change removes reading of xl.meta
as well as an additional lock, the performance gain
should increase 3x theoretically for

- PutObjectRetention
- PutObjectLegalHold

This optimization is mainly for Veeam like
workloads that require a certain level of iops
from these API calls, we were losing iops.
2021-10-30 08:22:04 -07:00
Aditya ManthramurthyandGitHub 3a7c79e2c7 Add new site replication feature (#13311)
This change allows a set of MinIO sites (clusters) to be configured 
for mutual replication of all buckets (including bucket policies, tags, 
object-lock configuration and bucket encryption), IAM policies, 
LDAP service accounts and LDAP STS accounts.
2021-10-06 16:36:31 -07:00
dd5804c10e Update PutObjectPart error message (#13313)
Co-authored-by: sinhaashish <ashish@minio.io>
2021-09-28 07:27:03 -07:00
Harshavardhana f9b104f37b Revert "s3: Put bucket tagging to return an error when bucket is not found (#13232)"
This reverts commit 91567ba916.

Revert because the error was incorrectly converted, there are
callers that rely on errConfigNotFound and it also took away
the migration code.

Instead the correct fix is PutBucketTaggingHandler() which
is already added.
2021-09-22 20:06:25 -07:00
Poorna KrishnamoorthyandGitHub 806b10b934 fix: improve error messages returned during replication setup (#13261) 2021-09-21 13:03:20 -07:00
HarshavardhanaandGitHub 5ed781a330 check for context canceled after competing for locks (#13239)
once we have competed for locks, verify if the
context is still valid - this is to ensure that
we do not start readdir() or read() calls on the
drives on canceled connections.
2021-09-17 14:11:01 -07:00
Anis ElleuchandGitHub 91567ba916 s3: Put bucket tagging to return an error when bucket is not found (#13232)
instead of creating new metadata in .minio.sys directory
2021-09-17 08:32:32 -07:00
HarshavardhanaandGitHub 5c448b1b97 fix: update allowed max chunk payloadsize to 16MiB (#13169)
fixes #13163
2021-09-09 14:37:07 -07:00
HarshavardhanaandGitHub 035882d292 fix: remove parentIsObject() check (#12851)
we will allow situations such as

```
a/b/1.txt
a/b
```

and

```
a/b
a/b/1.txt
```

we are going to document that this usecase is
not supported and we will never support it, if
any application does this users have to delete
the top level parent to make sure namespace is
accessible at lower level.

rest of the situations where the prefixes get
created across sets are supported as is.
2021-08-03 13:26:57 -07:00
Poorna KrishnamoorthyandGitHub d00783c923 Use rate.Limiter for bandwidth monitoring (#12506)
Bonus: fixes a hang when bandwidth caps are enabled for
synchronous replication
2021-06-24 18:29:30 -07:00
Poorna KrishnamoorthyandGitHub dbea8d2ee0 Add support for existing object replication. (#12109)
Also adding an API to allow resyncing replication when
existing object replication is enabled and the remote target
is entirely lost. With the `mc replicate reset` command, the
objects that are eligible for replication as per the replication
config will be resynced to target if existing object replication
is enabled on the rule.
2021-06-01 19:59:11 -07:00
HarshavardhanaandGitHub 1f262daf6f rename all remaining packages to internal/ (#12418)
This is to ensure that there are no projects
that try to import `minio/minio/pkg` into
their own repo. Any such common packages should
go to `https://github.com/minio/pkg`
2021-06-01 14:59:40 -07:00