Amazon S3

When using Amazon AWS, the easiest method to handle files is keeping it an S3 bucket that handles versioning, backups, storage-scale, etc. This allows easy access to the data and also a pay-as-use model to use S3's elastic storage model.

To use this, we can use the fs-s3fs filesystem available as part of pyfilesystem2. For this, the module fs-s3fs needs to be installed on the components:

• API Server
• API - Celery Worker

PyPI Link: https://pypi.org/project/fs-s3fs/

Installation

Warning

This will require internet connection

$INSTALL_DIR/venv-api/bin/pip install fs-s3fs

Configuration

In the API configurations, the following configurations need to be set:

• FS_URI = 's3://<access_key>:<secret_key>@<bucketname>/<bucketprefix>
  Needs to be set to be able to connect to the Database.
• Also, check the other configurations that may be required as described in API - Configurations > SQLAlchemy section