[PYTHON] [Django] A story about getting stuck in a swamp trying to validate a zip with form [TDD]

Django "UnicodeDecodeError:'utf-8' codec can't decode byte 0xbb in position 10: invalid start byte"

I got this error when I tried to upload and save a zip containing multiple files in the FileField of the form. From the conclusion, it seems that the data format that was thrown at the time of the test was bad.

However, I do not understand that it is difficult to reproduce the error because it clears for some reason when I try to reproduce the error after passing through. I was really in trouble ...

solution

When you first get an error,

tests.py



self.zip_error_test = File(open('tdd.zip'))
file_data = {'upload_train_file': uploadedFile }
form = DocumentForm(file_data)
form.is_valid()

I fold it, but it looks like this. This makes django angry. By the way, I was similarly angry with the ZIP that consolidated one file.

I solved it by changing it as below.

tests.py


with open("tdd.zip", mode="rb") as f:
   uploadedFile = InMemoryUploadedFile(f, "field", "tdd.zip", "application/x-zip-compressed", 100, None, content_type_extra={})
   file_data = {'field': uploadedFile }
   form = DocumentForm(request, data, file_data)
   self.assertFalse(form.is_valid(), f"error:{form.errors}/ZIP with multiple files was not validated")

Yes. This will work. You did it.

What was the cause?

I thought about various things. I wrote it in the custom validation of forms.py in the first place, so I wondered if that was the cause. I wondered if it wasn't good to bring cleaned_data, so I played with to_python (). I was wondering if I could save the file with forms.py.

The result of trying to think what would happen if I actually uploaded it from an HTML form instead of a test suddenly

I passed. </ b>

I thought that this was different from the data received by form, so I compared the types of data received by views and tests. Then

forms.py


def clean_field(self):
    print(type(self.cleaned_data['field']))

# views > django.core.files.uploadedfile.InMemoryUploadedFile
# tests > django.forms.widgets.ClearableFileInput

Wow, it ’s different.

Because of this, I changed the upload for the test to the one mentioned above.

This is one case. Thanks to that, I feel that I have become more familiar with Form. It's the so-called complete understanding. It's Kanpeki.

Summary

I tried to write this article after solving it, and when I wrote the code that would cause an error again, it cleared for some reason. Seriously a mystery. I wrote that it passes above

tests.py


with open("tdd.zip", mode="rb") as f:
   file_data = {'field': f}
   form = DocumentForm(request, data, file_data)
   self.assertFalse(form.is_valid(), f"error:{form.errors}/ZIP with multiple files was not validated")

This will still pass. What is it? Didn't you pass? Anything will pass without the mode =" rb ". Why is that? Did you get an error? It's true. Shinji.

I'm using the code above, thinking that it might not work again if I restart it. If you change the class explicitly, there will be no problem.

So it was a solution to the phenomenon that zip could not be saved for some reason.

Perhaps next week or so, I will summarize the form knowledge I gained in this matter. I expected to have.

Recommended Posts

[Django] A story about getting stuck in a swamp trying to validate a zip with form [TDD]
A story about trying to use cron on a Raspberry Pi and getting stuck in space
A story about trying to implement a private variable in Python.
[Note] A story about trying to override a class method with two underscores in Python 3 series.
A story about implementing a login screen with django
A story that required preparation when trying to do a Django tutorial with plain centos7
I got stuck when trying to specify a relative path with relative_to () in python
A story about trying to introduce Linter in the middle of a Python (Flask) project
A story about how to specify a relative path in python.
A story about competing with a friend in Othello AI Preparation
A story about how to deal with the CORS problem
A story about a python beginner stuck with No module named'http.server'
A story about a beginner participating in a project by Django from team building to product release in 6 weeks
A story about trying to automate a chot when cooking for yourself
A story about adding a REST API to a daemon made with Python
A story about trying to run multiple python versions (Mac edition)
[Django] How to test Form [TDD]
A story about trying to improve the testing process of a system written in C language for 20 years
[Small story] How to save matplotlib graphs in a batch with Jupyter
[Memorandum] A story about trying OpenCV tutorial (face recognition) in a Windows environment
A story about trying to connect to MySQL using Heroku and giving up
A story about a beginner trying hard to set up CentOS 8 (procedure memo)
A story about machine learning with Kyasuket
A story about trying to install uwsgi on an EC2 instance and failing
[Note] A story about not being able to break through a proxy with pip
A confusing story with two ways to implement XGBoost in Python + overall notes
[Django] Create a form that automatically fills in the address from the zip code
Try to create a Todo management site using WebSocket with Django (Swamp Dragon)
Dynamically add fields to Form objects in Django
[Python3] A story stuck with time zone conversion
How to develop a cart app with Django
Start Django in a virtual environment with Pipenv
A story stuck with handling Python binary data
Story of trying to use tensorboard with pytorch
A story packed with absolute values in numpy.ndarray
Configure a module with multiple files in Django
How to create a Rest Api in Django
A story that failed when trying to remove the suffix from the string with rstrip
A story that got stuck when trying to upgrade the Python version on GCE
A story about how Windows 10 users created an environment to use OpenCV3 with Python 3.5
How to use fixture in Django to populate sample data associated with a user model
A story of trial and error trying to create a dynamic user group in Slack
A story about a Python beginner trying to get Google search results using the API
I want to get an error message in Japanese with django Password Change Form
A less likely misunderstanding about how to specify a handler for a zip uploaded with Lambda
A story that I wanted to do a function like before_action used in rails with django [Beginner learns python with a reference book]