[PYTHON] Django Changed to save lots of data in one go

Since I saved nearly 30 daily tasks from TODO Plan, I had to access the database hundreds of times by creating regular tasks, and I wanted to reduce instability and load on the database.

Even so, I don't understand what's wrong with the database because it's accessed a lot ... To be clear, I don't know anything other than speed (laughs).

This implementation is

It is an implementation method called bulk_create. Referenced page: https://tsukasa-blog.com/programming/bulk-create-update/

As a result, the access to the database was reduced to 2 times (⌒∇⌒)

todo/views.py



def todoplancreatefunc(request):
    if request.method == 'GET':
        return render(request,'todo/todoplancreate.html')
    else:
        yearmonth = request.POST['yearmonth']
        year = int(yearmonth[0:4])
        month= int(yearmonth[5:7])
        lastday = calendar.monthrange(year, month)[1]
        new_object_list =[]

        #Process the set amount every day
        todo_plan = Todo_PlanModel.objects.order_by("hyoujijyun")
        for todoplan in todo_plan:
            if todoplan.cycle == "daily":
            #Process the set amount every day
                for i in range(lastday):
                    sakuseiday = datetime.date(year,month,i + 1)
                    new_object = TodoModel(
                        title = todoplan.title,
                        memo = todoplan.memo,
                        priority = todoplan.priority,
                        shisetsu_name = todoplan.shisetsu_name,
                        user = todoplan.user,
                        plants_startdate = sakuseiday,
                        plants_enddate = sakuseiday,
                        )
                    new_object_list.append(new_object)
            elif todoplan.cycle == "everyweek":
            #Process weekly
                for i in range(5):
                    day = get_day_of_nth_dow(year, month, i + 1, int(todoplan.youbi))
                    if day != None:
                        sakuseiday = datetime.date(year, month, day)
                        new_object = TodoModel(
                            title = todoplan.title,
                            memo = todoplan.memo,
                            priority = todoplan.priority,
                            shisetsu_name = todoplan.shisetsu_name,
                            user = todoplan.user,
                            plants_startdate = sakuseiday,
                            plants_enddate = sakuseiday,
                        )
                        new_object_list.append(new_object)
            
            elif todoplan.cycle == "monthly":
            #Process monthly
                #Get the end of the target month
                lastday = calendar.monthrange(year, month)[1]
                #Replace if the specified date is greater than the date at the end of the month
                if todoplan.day > lastday:
                    day = lastday
                else:
                    day = todoplan.day
                sakuseiday = datetime.date(year, month, day)
                new_object = TodoModel(
                    title = todoplan.title,
                    memo = todoplan.memo,
                    priority = todoplan.priority,
                    shisetsu_name = todoplan.shisetsu_name,
                    user = todoplan.user,
                    plants_startdate = sakuseiday,
                    plants_enddate = sakuseiday,
                    )
                new_object_list.append(new_object)

            elif todoplan.cycle == "day":
                #Process month / date specification
                #Get the end of the target month
                lastday = calendar.monthrange(year, int(todoplan.month))[1]
                #Replace if the specified date is greater than the date at the end of the month
                if todoplan.day > lastday:
                    day = lastday
                else:
                    day = todoplan.day
                sakuseiday = datetime.date(year, int(todoplan.month), day)
                new_object = TodoModel(
                    title = todoplan.title,
                    memo = todoplan.memo,
                    priority = todoplan.priority,
                    shisetsu_name = todoplan.shisetsu_name,
                    user = todoplan.user,
                    plants_startdate = sakuseiday,
                    plants_enddate = sakuseiday,
                    )
                new_object_list.append(new_object)

            elif todoplan.cycle == "week":
                #Designated on weekdays
                day = get_day_of_nth_dow(year, month, todoplan.week, int(todoplan.youbi))
                if day == None:
                    day = lastday = calendar.monthrange(year, month)[1]
                sakuseiday = datetime.date(year, month, day)
                new_object = TodoModel(
                    title = todoplan.title,
                    memo = todoplan.memo,
                    priority = todoplan.priority,
                    shisetsu_name = todoplan.shisetsu_name,
                    user = todoplan.user,
                    plants_startdate = sakuseiday,
                    plants_enddate = sakuseiday,
                    )
                new_object_list.append(new_object)
        TodoModel.objects.bulk_create(new_object_list)
        messages = 'Processing is finished'
        context = {
            'messages': messages,
            }
        return render(request,'todo/todoplancreate.html', context)

First, it is extracted from the table, and finally, it is saved in bulk with bule_create.

There is no doubt that this has made it faster (⌒∇⌒)

Recommended Posts

Django Changed to save lots of data in one go
Save multiple models in one form with Django
How to check ORM behavior in one file with django
Save in Japanese to StringProperty in Google App Engine data store
How to get an overview of your data in Pandas
Save the results of crawling with Scrapy to the Google Data Store
How to create a large amount of test data in MySQL? ??
Create a function to get the contents of the database in Go
Errors related to memcached in django
Implementation of login function in Django
How to convert 0.5 to 1056964608 in one shot
Try to put data in MongoDB
Post to slack in Go language
How to reflect CSS in Django
Use Cloud Dataflow to dynamically change the destination according to the value of the data and save it in GCS
Convert PDF of available stores of Go To EAT in Kagoshima prefecture to CSV
How to send a visualization image of data created in Python to Typetalk
Memory-saving conversion of log data to sequential category features in chronological order
Convert PDF of Go To EAT member stores in Ishikawa prefecture to CSV
How to deal with garbled characters in json of Django REST Framework
How to make only one data register on the Django admin screen
How to check the version of Django
How to delete expired sessions in Django
Conversion of time data in 25 o'clock notation
Hello world instead of localhost in Django
How to do Server-Sent Events in Django
[Introduction to Data Scientists] Basics of Python ♬
Books on data science to read in 2020
How to get people to try out django rest framework features in one file
Data analysis in Python Summary of sources to look at first for beginners
How to achieve something like a list of void * (or variant) in Go?
How to uniquely identify the source of access in the Django Generic Class View
How to count the number of elements in Django and output to a template
Convert PDF of list of Go To EAT member stores in Niigata prefecture to CSV
I'm addicted to the difference in how Flask and Django receive JSON data
Try to display the railway data of national land numerical information in 3D