Object Storage (S3-Compatible)
Buckets
DevZero's buckets allow storing a group of objects (with an API compatible with AWS S3). Buckets are regional in nature and can be access by any workspace in that region.
Creating a Bucket
Bucket names must be unique across all of DevZero, so we generate bucket names from a prefix you provide
In your default region
$ dz storage bucket create --prefix=my-fancy-bucket
ID Name Region Status
bucket-481692276ad14fcf96a32e176d597731 my-fancy-bucket-481692276ad14fcf96a32e176d597731 Portland, USA Creating
In an arbitrary region
$ dz storage bucket create --prefix=other-fancy-bucket --region=eu-north-1
ID Name Region Status
bucket-eaed65a5ff9a4a0284d28ad61f962f5f other-fancy-bucket-eaed65a5ff9a4a0284d28ad61f962f5f Stockholm, Sweden Creating
Listing your buckets
$ dz storage bucket list
ID Name Region Status
bucket-eaed65a5ff9a4a0284d28ad61f962f5f my-fancy-bucket-eaed65a5ff9a4a0284d28ad61f962f5f Portland, USA Available
bucket-481692276ad14fcf96a32e176d597731 other-fancy-bucket-481692276ad14fcf96a32e176d597731 Stockholm, Sweden Available
Accessing a bucket from a workspace with AWS cli
Install the cli
First, you need to install the AWS cli inside your workspace. You can also include that as a step in your recipe:
version: "3"
build:
steps:
- type: apt-get
packages: ["curl", "unzip"]
- type: command
user: devzero
command: |
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
rm -rf ./aws
rm -f awscliv2.zip
The object store credentials
If you launch your workspace after the bucket is created, you can find the credentials inside your .aws
directory.
$ dz workspace connect dear-filly-ubhg
...
...
$ ls .aws
config credentials
$ cat .aws/config
[profile devzero]
region = us-east-1
endpoint_url = http://object-store.devzero.svc
$ cat .aws/credentials
[devzero]
aws_access_key_id = XEZDB3UJ6X7HVBE7X7MA
aws_secret_access_key = 7yGIZON7EhFORz0I40BFniML36D2rl8CQQ5kXU6l
You can also view the object store credentials by running
$ dz storage bucket credentials
Endpoint URL: http://object-store.devzero.svc
Access Key ID: XEZDB3UJ6X7HVBE7X7MA
Secret Access Key: 7yGIZON7EhFORz0I40BFniML36D2rl8CQQ5kXU6l
And you can update the aws config by running
$ dz storage bucket credentials --update-aws-config
Endpoint URL: http://object-store.devzero.svc
Access Key ID: XEZDB3UJ6X7HVBE7X7MA
Secret Access Key: 7yGIZON7EhFORz0I40BFniML36D2rl8CQQ5kXU6l
Updated aws config with credentials
Using the API
You can now use the S3-compatible API with the the devzero
profile.
$ aws s3 --profile devzero ls
2024-10-24 02:27:12 my-fancy-bucket-eaed65a5ff9a4a0284d28ad61f962f5f
2024-10-11 18:18:15 other-fancy-bucket-481692276ad14fcf96a32e176d597731
$ dd if=/dev/zero of=my-object.dat bs=24M count=1
1+0 records in
1+0 records out
25165824 bytes (25 MB, 24 MiB) copied, 0.0455256 s, 553 MB/s
$ aws s3 --profile devzero cp my-object.dat s3://my-fancy-bucket-eaed65a5ff9a4a0284d28ad61f962f5f
upload: ./my-object.dat to s3://my-fancy-bucket-eaed65a5ff9a4a0284d28ad61f962f5f/my-object.dat
$ aws s3 --profile devzero ls s3://my-fancy-bucket-eaed65a5ff9a4a0284d28ad61f962f5f
2024-10-24 04:17:32 25165824 my-object.dat
$ aws s3 --profile devzero rm s3://my-fancy-bucket-eaed65a5ff9a4a0284d28ad61f962f5f/my-object.dat
delete: s3://my-fancy-bucket-eaed65a5ff9a4a0284d28ad61f962f5f/my-object.dat