Django S3 Not Uploading File to S3

File upload and download are some of the most performed actions on the web. With the rise of cloud-based solutions, companies are moving from an on-premises solution to cloud storage. A couple reasons being its cheaper cost and convenience.

This article will demonstrate how to upload user-generated files to Amazon S3 using the Django Residual Framework.

Table of contents

1. Prerequisites

2. What is AWS S3?

3. Building a simple Django Residue API application

4. Integrating AWS S3 into the Django Rest API application

5. Summary

Prerequisites

To follow this article along it would be helpful if the reader is comfortable working with Django and Django Rest Framework.

What is AWS S3?

AWS S3 is an acronym for Amazon Web Services Simple Storage Service (AWS S3). It is a deject-based service by Amazon for object storage.

Object storage is a blazon of storage where items are processed equally a data object. Reverse to the traditional method of storing files in the file arrangement hierarchy.

In the traditional file system, the basic unit of storage is a "file". In AWS S3, the basic unit of storage is chosen a "saucepan".

The AWS console and available SDKs from AWS are used to access buckets. These SDKs come up in supported popular languages such as Python and PHP.

There are several advantages of using AWS S3.

These includes:

• Scalability
• High functioning
• Audit capability
• Security
• Cost-effective
• About 99.999% availability (uptime)

AWS S3 can be used as a backup and disaster recovery tool likewise as in data analytics.

In this guide, we volition upload user-generated files using the Django Residual Framework.

Building a simple Django Rest API awarding

Nosotros are going to create a new Django project named Dropboxer. Dropboxer is a uncomplicated file storage application. We can observe the complete source code for this project in this repository.

Execute the commands below to prepare the project.

            pip install django pip install djangorestframework django-admin startproject dropboxer cd dropboxer python manage.py startapp uploader                      

The binder construction looks like the structure below:

            - dropboxer - uploader - manage.py                      

Add the new app to INSTALLED_APPS in settings.py:

                          …              # Application definition              INSTALLED_APPS              =              [              'django.contrib.admin',              'django.contrib.auth',              'django.contrib.contenttypes',              'django.contrib.sessions',              'django.contrib.messages',              'django.contrib.staticfiles',              # 3rd political party apps              'rest_framework',              # local apps              'uploader',              # new              ]              …              # Static files (CSS, JavaScript, Images)              # https://docs.djangoproject.com/en/2.two/howto/static-files/              STATIC_URL              =              '/static/'              STATIC_ROOT              =              os.path.bring together(BASE_DIR,              'static') MEDIA_URL              =              '/media/'              MEDIA_ROOT              =              os.path.join(BASE_DIR,              'media')                      

Add the code snippet beneath to urls.py file in the dropboxer project directory.

                          from              django.conf              import              settings              # new              from              django.conf.urls.static              import              static              # new              from              django.contrib              import              admin              from              django.urls              import              path urlpatterns              =              [     path('admin/', admin.site.urls), ]              if              settings.DEBUG:     urlpatterns              +=              static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)     urlpatterns              +=              static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)                      

Create serializers.py and urls.py files in the uploader app.

In models.py file, we create a simple model that represents a single file.

                          from              django.db              import              models              course              DropBox(models.Model):     title              =              models.CharField(max_length=              30)     document              =              models.FileField(max_length=              30)     created_at              =              models.DateTimeField(auto_now_add=Truthful)     updated_at              =              models.DateTimeField(auto_now=True)              class              Meta:         verbose_name_plural              =              'Driblet Boxes'                      

Add the code snippet below in serializers.py file:

                          from              rest_framework              import              serializers              from              .models              import              DropBox              class              DropBoxSerializer(serializers.ModelSerializer):              form              Meta:         model              =              DropBox         fields              =              '__all__'                      

In uploader/views.py:

                          from              rest_framework              import              viewsets, parsers              from              .models              import              DropBox              from              .serializers              import              DropBoxSerializer              class              DropBoxViewset(viewsets.ModelViewSet):       queryset              =              DropBox.objects.all()     serializer_class              =              DropBoxSerializer     parser_classes              =              [parsers.MultiPartParser, parsers.FormParser]     http_method_names              =              ['get',              'post',              'patch',              'delete']                      

In uploader/urls.py:

                          from              rest_framework.routers              import              SimpleRouter              from              .views              import              DropBoxViewset router              =              SimpleRouter() router.annals('accounts', DropBoxViewset) urlpatterns              =              router.urls                      

In uploader/admin:

                          from              django.contrib              import              admin              from              .models              import              DropBox admin.site.register(DropBox)                      

In dropboxer/urls.py:

                          ...              from              django.urls              import              path, include              # new              urlpatterns              =              [     path('admin/', admin.site.urls),     path('api/', include('rest_framework.urls')),              # new              path('', include('uploader.urls')),              # new              ]                      

To create a table in the database from the model that nosotros created above, execute the command below.

            python manage.py makemigrations python manage.py migrate                      

Start the evolution server by executing the command below:

            python manage.py runserver                      

On the browser navigate to accounts. We find the sample file uploaded in the media folder.

This sample file was created when we uploaded it via the API on the browser as shown below:

Integrating AWS S3 into the Django API application

We accept a working application API endpoint. You need an AWS business relationship to enable the integration of AWS S3 into your Django awarding.

Sign up if yous exercise not already accept an business relationship, sign in if you have an existing AWS account.

Search for S3:

Click on "create bucket" push button:

Provide a unique proper name for your S3 bucket that is globally identified:

Naming an AWS S3 bucket may take some trial and error earlier a proper noun that does non already exist is discovered.

Go along default option, click create bucket:

You will be redirected to the AWS S3 console which now shows the newly created bucket:

We accept successfully created an AWS S3 bucket. Recall during saucepan creation, public access to the S3 saucepan was blocked.

To access the created bucket from our application, we volition need to gain access using AWS IAM. AWS IAM is an acronym for Identity and Admission Direction.

It is used to provide access to rights and privileges on AWS resource. Currently, we tin can only access the S3 saucepan through the console.

AWS allows access to its resources such equally AWS S3 through User and Roles. You can read more almost how AWS does this here

Search for IAM using AWS search bar:

Click User on the IAM side carte du jour:

Click Add together user on the IAM User dashboard:

Provide a user name and check the programmatic access box:

In Ready Permissions, choose "Adhere existing policies direct" and check AWSS3FullAcess box:

Click through and review your choice earlier creating the user:

On successful creation, information technology generated an AWS Access and Secret primal:

Shop the AWS Underground Key before finishing because the Hush-hush Key won't be shown again.

Once done, nosotros can view the newly created AWS user on the IAM User dashboard:

In this commodity, we will use Django-storages to connect to the AWS S3 bucket.

Django-storages is a collection of custom storage backends for Django framework. We volition use the AWS S3 integration from the collection in Django-storages package.

Install django-storages into the Django application via pip:

            pip install django-storages                      

In settings.py add together the code snippet beneath:

                          ...              AWS_ACCESS_KEY_ID              =              <YOUR AWS ACCESS KEY>              AWS_SECRET_ACCESS_KEY              =              <YOUR AWS SECRET KEY>              AWS_STORAGE_BUCKET_NAME              =              <YOUR AWS S3 Bucket NAME>              AWS_S3_SIGNATURE_VERSION              =              's3v4'              AWS_S3_REGION_NAME              =              <YOUR AWS S3 BUCKET LOCATION>              AWS_S3_FILE_OVERWRITE              =              False AWS_DEFAULT_ACL              =              None AWS_S3_VERIFY              =              True DEFAULT_FILE_STORAGE              =              'storages.backends.s3boto3.S3Boto3Storage'                      

AWS_ACCESS_KEY_ID: is the key identifier of the IAM User. It starts with "AK****"

AWS_SECRET_ACCESS_KEY: is the generated 40 alphanumeric characters.

AWS_S3_REGION_NAME: refers to the AWS Region in the S3 console dashboard. For case: us-east-ane, eu-west-ii.

AWS_S3_SIGNATURE_VERSION: is the version of the signature used for generating pre-signed URLs. AWS S3 buckets need the signature to grant admission.

AWS_S3_FILE_OVERWRITE: when set to True, AWS S3 overwrites a file with the same name and format. If ready to Simulated, AWS appends unique strings to the newly uploaded file proper noun. It does not override the existing file.

Restart the development server past executing the command below:

            python manage.py runserver                      

On the browser, navigate to localhost and retry uploading a sample file:

Click the link in the certificate field. Observe the link now has "s3.amazon**". You will exist able to access the file. Here, the uploaded file is titled "Large O Cheatsheet":

Summary

In this article, nosotros created an AWS S3 bucket and assigned IAM User with full access to the AWS S3 saucepan. We uploaded files to the AWS S3 bucket using Access Central and AWS Secret Primal from our Django Rest API application.

Happy coding!

References

• Django-storages documentation

---

Peer Review Contributions by: Odhiambo Paul

slaterablat1971.blogspot.com

Source: https://www.section.io/engineering-education/how-to-upload-files-to-aws-s3-using-django-rest-framework/

Share this post