文档基于2025-10-10日社区最终版构建     
切换到英文版    进入演示系统    进入交流社区

MinIO Documentation

mc batch generate

MinIO 版本变更说明:RELEASE.2022-10-08T20-11-00Z 或更高版本

语法

Themc batch generate该命令为指定的作业类型创建一个基本的 YAML 格式模板文件。

在 MinIO 创建文件后,使用您偏好的文本编辑器工具打开该文件以进行进一步自定义。 您可以在每个批处理文件中定义一个作业任务定义。

工作类型对于支持的作业,您可以生成。

以下命令为复制作业创建一个基本的 YAML 文件mybucketbucket of themyminio别名。

mc batch generate myminio replicate

该命令具有以下语法:

mc [GLOBALFLAGS] batch generate \
                       ALIAS   \
                       JOBTYPE
  • Brackets[]表示可选参数。

  • 共享同一行的参数是相互依赖的。

  • 使用竖线分隔的参数|操作符是互斥的。

将示例复制到文本编辑器中,根据需要修改,然后在终端/Shell中运行命令。

参数

ALIAS
必需

The别名用于生成 YAML 模板文件。 指定的alias不限制您可以使用生成文件的部署环境。

例如:

mc batch generate myminio replicate
JOBTYPE
必需

要为其生成 YAML 文档的作业类型。

支持以下值:

全局标志

此命令支持任何全局标志.

示例

生成一个yaml为 Replicate 作业类型创建文件

以下命令生成一个复制类型批处理作业的 YAML 蓝图,并将文件命名为replicate随着.yaml扩展:

mc batch generate alias replicate > replicate.yaml
  • 替换alias随着alias用于生成 yaml 文件。

  • 替换replicate根据要生成的作业类型生成一个 YAML 文件。

    mc:mc batch支持replicatekeyrotate工作类型。

S3 Compatibility

Themc命令行工具专为兼容 AWS S3 API 而构建,并已通过 MinIO 和 AWS S3 测试,确保功能与行为符合预期。

MinIO 不提供对其他 S3 兼容服务的保证,因为它们的 S3 API 实现方式未知,因此不受支持。尽管mc命令可能按文档所述工作,任何此类使用风险自负。

职位类型

mc batch目前支持以下作业任务类型:

  • 复制

    在两个 MinIO 部署之间复制对象。 提供类似于存储桶复制作为批处理作业而非持续扫描功能。

  • 密钥轮换

    MinIO 版本新增功能:RELEASE.2023-04-07T05-28-58Z

    轮换 MinIO 部署中静态对象的 SSE-S3 或 SSE-KMS 密钥。

  • 过期

    MinIO 版本新增功能:RELEASE.2023-12-02T10-51-33Z

    基于类似语义使对象过期自动对象过期.

replicate

您可以使用以下示例配置作为构建自定义复制批处理作业的起点:

replicate:
  apiVersion: v1
  # source of the objects to be replicated
  source:
    type: TYPE # valid values are "s3" or "minio"
    bucket: BUCKET
    prefix: PREFIX # 'PREFIX' is optional
    # If your source is the 'local' alias specified to 'mc batch start', then the 'endpoint' and 'credentials' fields are optional and can be omitted
    # Either the 'source' or 'remote' *must* be the "local" deployment
    endpoint: "http[s]://HOSTNAME:PORT" 
    # path: "on|off|auto" # "on" enables path-style bucket lookup. "off" enables virtual host (DNS)-style bucket lookup. Defaults to "auto"
    credentials:
      accessKey: ACCESS-KEY # Required
      secretKey: SECRET-KEY # Required
    # sessionToken: SESSION-TOKEN # Optional only available when rotating credentials are used
    snowball: # automatically activated if the source is local
      disable: false # optionally turn-off snowball archive transfer
      batch: 100 # upto this many objects per archive
      inmemory: true # indicates if the archive must be staged locally or in-memory
      compress: false # S2/Snappy compressed archive
      smallerThan: 5MiB # create archive for all objects smaller than 5MiB
      skipErrs: false # skips any source side read() errors

  # target where the objects must be replicated
  target:
    type: TYPE # valid values are "s3" or "minio"
    bucket: BUCKET
    prefix: PREFIX # 'PREFIX' is optional
    # If your source is the 'local' alias specified to 'mc batch start', then the 'endpoint' and 'credentials' fields are optional and can be omitted

    # Either the 'source' or 'remote' *must* be the "local" deployment
    endpoint: "http[s]://HOSTNAME:PORT"
    # path: "on|off|auto" # "on" enables path-style bucket lookup. "off" enables virtual host (DNS)-style bucket lookup. Defaults to "auto"
    credentials:
      accessKey: ACCESS-KEY
      secretKey: SECRET-KEY
    # sessionToken: SESSION-TOKEN # Optional only available when rotating credentials are used

  # NOTE: All flags are optional
  # - filtering criteria only applies for all source objects match the criteria
  # - configurable notification endpoints
  # - configurable retries for the job (each retry skips successfully previously replaced objects)
  flags:
    filter:
      newerThan: "7d" # match objects newer than this value (e.g. 7d10h31s)
      olderThan: "7d" # match objects older than this value (e.g. 7d10h31s)
      createdAfter: "datetime" # match objects created after this date and time in RFC3339 format
      createdBefore: "datetime" # match objects created before this date and time in RFC3339 format

      ## NOTE: tags are not supported when "source" is remote.
      # tags:
      #   - key: "name"
      #     value: "pick*" # match objects with tag 'name', with all values starting with 'pick'

      # metadata:
      #   - key: "content-type"
      #     value: "image/*" # match objects with 'content-type', with all values starting with 'image/'

    notify:
      endpoint: "https://notify.endpoint" # notification endpoint to receive job status events
      token: "Bearer xxxxx" # optional authentication token for the notification endpoint

    retry:
      attempts: 10 # number of retries for the job before giving up
      delay: "500ms" # least amount of delay between each retry

复制批处理作业参考有关每个键的更多完整文档。

keyrotate

您可以使用以下示例配置作为构建自定义密钥轮换批处理作业的起点:

keyrotate:
  apiVersion: v1
  bucket: BUCKET
  prefix: PREFIX
  encryption:
    type: sse-s3 # valid values are sse-s3 and sse-kms
    key: <new-kms-key> # valid only for sse-kms
    context: <new-kms-key-context> # valid only for sse-kms

  # optional flags based filtering criteria
  # for all objects
  flags:
    filter:
      newerThan: "7d" # match objects newer than this value (e.g. 7d10h31s)
      olderThan: "7d" # match objects older than this value (e.g. 7d10h31s)
      createdAfter: "date" # match objects created after this date and time in RFC3339 format
      createdBefore: "date" # match objects created before this date and time in RFC3339 format
      tags:
        - key: "name"
          value: "pick*" # match objects with tag 'name', with all values starting with 'pick'
      metadata:
        - key: "content-type"
          value: "image/*" # match objects with 'content-type', with all values starting with 'image/'
      kmskey: "key-id" # match objects with KMS key-id (applicable only for sse-kms)
    notify:
      endpoint: "https://notify.endpoint" # notification endpoint to receive job status events
      token: "Bearer xxxxx" # optional authentication token for the notification endpoint
    retry:
      attempts: 10 # number of retries for the job before giving up
      delay: "500ms" # least amount of delay between each retry

密钥轮换批处理作业参考有关每个键的更多完整文档。

expire

您可以使用以下示例配置作为构建自定义过期批处理作业的起点:

expire:
  apiVersion: v1
  bucket: mybucket # Bucket where this job will expire matching objects from
  prefix: myprefix # (Optional) Prefix under which this job will expire objects matching the rules below.
  rules:
    - type: object  # objects with zero ore more older versions
      name: NAME # match object names that satisfy the wildcard expression.
      olderThan: 70h # match objects older than this value
      createdBefore: "2006-01-02T15:04:05.00Z" # match objects created before this date and time in RFC3339 format
      tags:
        - key: name
          value: pick* # match objects with tag 'name', all values starting with 'pick'
      metadata:
        - key: content-type
          value: image/* # match objects with 'content-type', all values starting with 'image/'
      size:
        lessThan: 10MiB # match objects with size less than this value (e.g. 10MiB)
        greaterThan: 1MiB # match objects with size greater than this value (e.g. 1MiB)
      purge:
          # retainVersions: 0 # (default) delete all versions of the object. This option is the fastest.
          # retainVersions: 5 # keep the latest 5 versions of the object.

    - type: deleted # objects with delete marker as their latest version
      name: NAME # match object names that satisfy the wildcard expression.
      olderThan: 10h # match objects older than this value (e.g. 7d10h31s)
      createdBefore: "2006-01-02T15:04:05.00Z" # match objects created before this date and time in RFC3339 format
      purge:
          # retainVersions: 0 # (default) delete all versions of the object. This option is the fastest.
          # retainVersions: 5 # keep the latest 5 versions of the object including delete markers.

  notify:
    endpoint: https://notify.endpoint # notification endpoint to receive job completion status
    token: Bearer xxxxx # optional authentication token for the notification endpoint

  retry:
    attempts: 10 # number of retries for the job before giving up
    delay: 500ms # least amount of delay between each retry

过期批处理作业参考有关每个键的更多完整文档。