Object Storage (S3-Compatible)

Buckets

DevZero's buckets allow storing a group of objects (with an API compatible with AWS S3). Buckets are regional in nature and can be access by any workspace in that region.

Creating a Bucket

Bucket names must be unique across all of DevZero, so we generate bucket names from a prefix you provide

In your default region

$ dz storage bucket create --prefix=my-fancy-bucket
ID                                       Name                                              Region          Status
bucket-481692276ad14fcf96a32e176d597731  my-fancy-bucket-481692276ad14fcf96a32e176d597731  Portland, USA   Creating

In an arbitrary region

$ dz storage bucket create --prefix=other-fancy-bucket --region=eu-north-1
ID                                       Name                                                 Region             Status
bucket-eaed65a5ff9a4a0284d28ad61f962f5f  other-fancy-bucket-eaed65a5ff9a4a0284d28ad61f962f5f  Stockholm, Sweden  Creating

Listing your buckets

$ dz storage bucket list
ID                                       Name                                                 Region             Status
bucket-eaed65a5ff9a4a0284d28ad61f962f5f  my-fancy-bucket-eaed65a5ff9a4a0284d28ad61f962f5f     Portland, USA      Available
bucket-481692276ad14fcf96a32e176d597731  other-fancy-bucket-481692276ad14fcf96a32e176d597731  Stockholm, Sweden  Available

Accessing a bucket from a workspace with AWS cli

Install the cli

First, you need to install the AWS cli inside your workspace. You can also include that as a step in your recipe:

version: "3"
build:
  steps:
    - type: apt-get
      packages: ["curl", "unzip"]
    - type: command
      user: devzero
      command: |
        curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
        unzip awscliv2.zip
        sudo ./aws/install
        rm -rf ./aws
        rm -f awscliv2.zip

The object store credentials

If you launch your workspace after the bucket is created, you can find the credentials inside your .aws directory.

$ dz workspace connect dear-filly-ubhg
...
...

$ ls .aws
config  credentials

$ cat .aws/config
[profile devzero]
region       = us-east-1
endpoint_url = http://object-store.devzero.svc

$ cat .aws/credentials
[devzero]
aws_access_key_id     = XEZDB3UJ6X7HVBE7X7MA
aws_secret_access_key = 7yGIZON7EhFORz0I40BFniML36D2rl8CQQ5kXU6l

You can also view the object store credentials by running

$ dz storage bucket credentials
Endpoint URL: http://object-store.devzero.svc
Access Key ID: XEZDB3UJ6X7HVBE7X7MA
Secret Access Key: 7yGIZON7EhFORz0I40BFniML36D2rl8CQQ5kXU6l

And you can update the aws config by running

$ dz storage bucket credentials --update-aws-config
Endpoint URL: http://object-store.devzero.svc
Access Key ID: XEZDB3UJ6X7HVBE7X7MA
Secret Access Key: 7yGIZON7EhFORz0I40BFniML36D2rl8CQQ5kXU6l
Updated aws config with credentials

Using the API

You can now use the S3-compatible API with the the devzero profile.

$ aws s3 --profile devzero ls
2024-10-24 02:27:12 my-fancy-bucket-eaed65a5ff9a4a0284d28ad61f962f5f
2024-10-11 18:18:15 other-fancy-bucket-481692276ad14fcf96a32e176d597731

$ dd if=/dev/zero of=my-object.dat  bs=24M  count=1
1+0 records in
1+0 records out
25165824 bytes (25 MB, 24 MiB) copied, 0.0455256 s, 553 MB/s

$ aws s3 --profile devzero cp my-object.dat s3://my-fancy-bucket-eaed65a5ff9a4a0284d28ad61f962f5f
upload: ./my-object.dat to s3://my-fancy-bucket-eaed65a5ff9a4a0284d28ad61f962f5f/my-object.dat

$ aws s3 --profile devzero ls s3://my-fancy-bucket-eaed65a5ff9a4a0284d28ad61f962f5f
2024-10-24 04:17:32   25165824 my-object.dat

$ aws s3 --profile devzero rm s3://my-fancy-bucket-eaed65a5ff9a4a0284d28ad61f962f5f/my-object.dat
delete: s3://my-fancy-bucket-eaed65a5ff9a4a0284d28ad61f962f5f/my-object.dat

Object Storage (S3-Compatible)

On this page