[PYTHON] I tried to create CSV upload, data processing, download function with Django

Thing you want to do

Import multiple CSVs, process them, and then return the results as CSV. This time we will implement this using the following case.

Case

The following CSV is available at a university. ʻAvailable.csvhas information on how much capacity each class has. reserved.csv` has information on how many students are planning to take classes in each class. This time, if you upload these files on the browser, you will create an application that downloads a CSV that shows how much space each class currently has.

design

On the front side, there are mainly the following two movements. ①: File upload (2): Based on the uploaded file, spit out CSV indicating the freeness of each class.

Premise

It's assumed that you already have a Django project folder (myproject) and an app folder (ʻapp`). If not, see here.

Folder structure

This time, the folder structure is as follows.

├─app
│  │  admin.py
│  │  apps.py
│  │  forms.py
│  │  functions.py
│  │  models.py
│  │  tests.py
│  │  views.py
│  │  __init__.py
│  │
│  ├─migrations
│  │    __init__.py
│  │
│  ├─static
│  │  └─upload
│  │          test.txt
│  │
│  └─templates
│       index.html
│
├─myproject
│  │  settings.py
│  │  urls.py
│  │  wsgi.py
│  │  __init__.py
│  │
│  └─static
│        humans.txt
│
└─staticfiles

Form creation

First, define the form in forms.py as follows so that you can upload the file.

from django import forms
#Define Classroom Form
class ClassroomForm(forms.Form):    
    availablity = forms.FileField() 
    reservation = forms.FileField()

Pass the form to HTML

Next, in order to display the created form on the front side, pass the form to HTML with views.py as shown below.

from django.shortcuts import render  
from app.forms import ClassroomForm  

def index(request): 
    ###Index ClassroomForm.Pass to html
    classroom = ClassroomForm()  
    return render(request,"index.html",{'form':classroom})

Now use {{form.as_p}} in ʻindex.html` and pull the form as follows:

<body>  
    <form method="POST" class="post-form" enctype="multipart/form-data">  
            {% csrf_token %}  
            {{ form.as_p }}  
            <button type="submit" class="save btn btn-default">Save</button>  
    </form>  
</body>

POST processing

If you submit the form as it is, the POSTed information will be passed to the ʻindex method of views.py. So, rewrite ʻindex as follows so that POST processing is performed.

from django.shortcuts import render  
from django.http import HttpResponse  
from app.functions import process_files
from app.functions import write_into_csv
from app.forms import ClassroomForm  
from django.template import loader
import csv

def index(request):  
    if request.method == 'POST':  
        classroom = ClassroomForm(request.POST, request.FILES)
        #If there is data in the classroom
        if classroom.is_valid():  
            availability = request.FILES['availablity']
            reservation = request.FILES['reservation']
            #implement process_files
            csv_data = process_files(availability, reservation)
            #download result as a csv format             
            response = write_into_csv(csv_data)

            return response
    else:       
        ###Index ClassroomForm.Pass to html
        classroom = ClassroomForm()  
        
        return render(request,"index.html",{'form':classroom})

Here, we are calling two methods, process_files and write_into_csv, defined in functions.py. The following defines this method.

Define logic

Define the logic used in views.py in functions.py.

import csv
from django.http import HttpResponse  

### Process csv files
def process_files(availability, reservation):  
    
        """ Description
        :type availability:
        :param availability:
    
        :type reservation:
        :param reservation:
    
        :raises:
    
        :rtype:
        """
        availability_dict = convert_to_dict(availability)
        reservation_dict = convert_to_dict(reservation)
    
        csv_data = []

        courses = list(availability_dict)
    
        for course in courses:
            remaining = availability_dict[course] - reservation_dict[course]
            row = [course, remaining]
            csv_data.append(row)
        return csv_data

def convert_to_dict(file):
    
        """ Description
        :type file:
        :param file:
    
        :raises:
    
        :rtype:
        """
        data = file.read().decode("utf-8-sig")
        lines = data.split("\r\n")
        dict = {}
        for line in lines:	
            fields = line.split(",")
            course = fields[0]
            capacity = int(fields[1])
            dict[course] = capacity

        return dict


def write_into_csv(csv_data):
        """ Description
        :type csv_data:
        :param csv_data:
    
        :raises:
    
        :rtype:
        """
        response = HttpResponse(content_type='text/csv')
        response['Content-Disposition'] = 'attachment; filename="download.csv"'
        writer = csv.writer(response)  

        for row in csv_data:
            writer.writerow(row)

        return response

With the above, the function as designed was implemented.

Demo

Finally

Please do Like if you find it helpful.

Github https://github.com/norifumi92/csv_uploader/tree/develop

reference

https://www.javatpoint.com/django-file-upload https://codepen.io/adamlaki/pen/VYpewx https://www.pythoncircle.com/post/30/how-to-upload-and-process-the-csv-file-in-django/ https://into-the-program.com/customize-input-type-file/ https://docs.djangoproject.com/en/3.0/topics/forms/