[PYTHON] I want to know more about individual developers. YouDev has been released.

eyecatch.png

YouDev has been released. 2020.5.01

YouDev is a service that focuses on individual developers and lets them know more (I want to know ...).

Even if the service is recognized, I think that the developer is not in the spotlight, so I think that individual developers can appeal not only the service but also the developer himself. I created it.

We are always looking for individual developers, so ** I am an individual developer! If you think **, please register!

In addition, we accept improvements regardless of whether they can be improved or not!

My specs

・ ** Heppoko Engineer </ font> ** ・ Development history: 3 years ・ Personal development history: Half a year (In 2020.01, I decided to do personal development this year! **) ・ Development language: python (half a year) ・ Design sense: ** None ** (Design is impossible without using a framework.) ・ I can't keep up with the speed of new technology these days. ・ Python is hot! ?? I'm just thinking ** ・ Let's post to Qiita when you start personal development! I thought, but the atmosphere was different. (There seems to be no curtain for Heppoko engineers.) ・ Qiita can only be written in MarkDown! ?? What is it impossible!

What YouDev is aiming for

Anyway, I wanted to know about individual developers.

What kind of people are individual developers and what kind of activities are they doing? A service that is useful when you think

Aim for a place for activities to increase awareness that is beneficial to individual developers.

Production environment

youdev構成図.png

Like this.

DB used MySQL because I hadn't used PostgreSQL very much. I really wanted to use AWS RDS, but since it is not a monetized service, I installed it quietly in EC2. But I still want to use RDS. ..

I wanted to avoid using S3 if possible, but I decided to use S3 because there is image data registered by the user. In the first place, the storage capacity of EC2 is only 8GB, so I chose S3 quietly here as well.

However, for image data, CSS, and script systems that are commonly used within the service, a new static folder is created on the EC2 side and distributed from there. (At worst, it's a file that can be corrupted.)

The source code is managed on private github.

Use Amazon S3 with Django

Immediately after the production environment started, I used to operate media files in the same directory as EC2 static, so I have to migrate them to S3 first. ..

Creating an S3 bucket

Seriously create a bucket for S3.

There were basically no stumbling points here.

  1. Create a bucket
  2. Add IAM user (maybe you should copy the access key here)
  3. Create group (policy applies Amazon S3 Full Access) Add the user created in 4.2 to the group created in 3.
  4. Add the user created in 2 to the bucket users

Transition to S3

First of all, confirm the authentication below

$ aws configure list

      Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile                <not set>             None    None
access_key                <not set>             None    None
secret_key                <not set>             None    None
    region                <not set>             None    None

If it is not set, you will not be able to connect to S3, so set the access key and secret key of the user you created earlier.

$ aws configure

AWS Access Key ID [None]: <access key>
AWS Secret Access Key [None]:<Secret key>
Default region name [None]: <I haven't entered anything>
Default output format [None]:<I haven't entered anything> 

Now that you can connect to S3, upload it!


$ cd /usr/share/nginx/html/media/ #This is the directory that contains the files you want to migrate

$ aws s3 cp --recursive uploads s3://<Bucket name>/<Folder name>

I got it!

By the way -If --recursive is added, directories will be targeted. -If sync is used instead of cp, it will be synchronized instead of copy.

Django settings

You have to upload the media file on Django side and change the reference destination to face S3.

Install the following modules to use S3 with Django

Installation


$ pip install django-storages
$ pip install boto3

Add the following to the setting

settings.py



INSTALLED_APPS = [
    ...
    'storages', #add to
]

...

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
                'social_django.context_processors.backends',
                'social_django.context_processors.login_redirect',
                
                'django.template.context_processors.media', #add to
                #By adding this, from template{{MEDIA_URL}}You will be able to refer to S3 with.
            ],
        },
    },
]

...

# MEDIA_URL = '/media/'
AWS_ACCESS_KEY_ID = '<access key>'
AWS_SECRET_ACCESS_KEY = '<Secret key>'
AWS_STORAGE_BUCKET_NAME = '<Bucket name>'

AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
AWS_LOCATION = 'media'
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
MEDIA_URL = "https://%s/%s/" % (AWS_S3_CUSTOM_DOMAIN, AWS_LOCATION)

Now you can upload to S3 and browse S3.

Library system used

social-auth-app-django

A library that allows you to log in to services created with Django using social services (twitter, google, facebook, etc.).

Authentication settings for twitter account

Installation


$ pip install social-auth-app-django

settings.py


INSTALLED_APPS = [
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'social_django', #Add here
]
...

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
                'social_django.context_processors.backends',#Add here
                'social_django.context_processors.login_redirect',#Add here
            ],
        },
    },
]

...

SOCIAL_AUTH_URL_NAMESPACE = 'social' #add to

#AUTHENTICATION_Added BACKENDS
AUTHENTICATION_BACKENDS = [
    'social_core.backends.twitter.TwitterOAuth',
    'django.contrib.auth.backends.ModelBackend',
]

#The following should be set in environment variables in a production environment!
SOCIAL_AUTH_TWITTER_KEY = 'xxxxxxxx' # Consumer Key (API Key)
SOCIAL_AUTH_TWITTER_SECRET = 'xxxxxxxxx'# Consumer Secret (API Secret)
SOCIAL_AUTH_LOGIN_REDIRECT_URL = '' #Redirect URL at login
LOGOUT_REDIRECT_URL = '' #Redirect URL when logging out

project/urls.py


urlpatterns = [
    ...
    path('', include('social_django.urls')), #add to#
    #By adding ↑, each process can be performed by accessing the following URL from the application side.
    #Login:/login/twitter
    #Log out:/disconnect/twitter/ 
    #* Log out is auth on the Django side_views.LogoutView.as_Better to use view
    #complete:/complete/twitter/
    ...   
]

/ complete / twitter / will be the URL of the twitter API Callback. Example) スクリーンショット 2020-05-12 22.04.03.png

After that, you can use it by migrating

migrate


$ python manage.py makemigrations social_django
$ python manage.py migrate

requests-oauthlib

This is a library that allows you to implement OAuth when using the Twitter API with a few codes.

Installation


$ pip install requests requests_oauthlib

settings.py



TWITTER_ACCESS_TOKEN = 'XXXXXX' #add to
TWITTER_ACCESS_TOKEN_SECRET='XXXXX' #add to

Here, as an example, the method of acquiring user information is described.

Fill in the above anywhere in settings.py.

views.py



CK = settings.SOCIAL_AUTH_TWITTER_KEY 
CS = settings.SOCIAL_AUTH_TWITTER_SECRET
AT = settings.TWITTER_ACCESS_TOKEN
ATS = settings.TWITTER_ACCESS_TOKEN_SECRET

def getAccount(request,screen_name):
    url = "https://api.twitter.com/1.1/users/show.json" #See official documentation for endpoints
    params = {
        'screen_name':screen_name
    }
    req = twitter.get(url, params = params)
    
    if req.status_code == 200:
        res = json.loads(req.text)
        
    else:
        res=""
        print("Failed: %d" % req.status_code)
    
    return res

Materialize CSS (design)

Official documentation

As I mentioned at the beginning, you can't do CSS without using the library. .. .. You can also install scss, so if you don't like it, you can change the main color etc. all at once.

** * Note: If the select box is defective and the select selection field overlaps the select box below. There was a problem that it could not be selected correctly. </ font> **

I was desperate, but the repair code was posted on the following github, so if you use this, it will be solved!

Repaired version of git hub

cropper.js

A library that crops images. Of course, I couldn't implement the trimming process myself, so I used it.

The implementation method was a mystery no matter how many times I read the document, but I finally understood the following site through dozens of times.

Even if you go through a few dozen times, you should know!

Most of the libraries used above.

Stumble

mysql version

From Django 3.0, the driver that connects to mysql is described in the official document as mysqlclient recommended, so I tried installing it quietly, but there was an error. ..

python


$ pip install mysqlclient

> ~
 check the manual that corresponds to your MySQL server version for the right syntax to use near 'rsion' at line 1
 ~

Mysterious error

Cause: Seems to be due to an older version of mysql
 

I am using AWS Cloud9 as the development environment, but with cloud9, mysql is installed in the initial state.

However, it seems that the version is 5.5 and mysqlclient can only be used from 5.7, so I removed 5.5 and installed 5.7.

python


 sudo yum remove mysql* #Remove mysql
 #(If you are careful, sudo yum-y remove mysql-config mysql55-server mysql55-libs mysql55)

 sudo yum -y install mysql57-server mysql57
 sudo yum -y install mysql-devel

Register emoji in mysql

Originally, charset was set to utf-8, so pictograms could not be registered.

Change configuration file

/etc/my.cnf


[mysqld]
character-set-server=utf8mb4

[client]
default-character-set=utf8mb4

Restart mysql

/etc/my.cnf


sudo service mysqld restart

Changed because the table was already created in utf-8

mysql


ALTER TABLE targettable CONVERT TO CHARACTER SET utf8mb4;

In the case of Django, settings are also required on the Django side

settings.py


DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'xxx',
        'USER': 'xxx',
        'PASSWORD': 'xxx',
        'HOST': '',
        'PORT': '',
        #Added OPTIONS
        'OPTIONS': {
            'charset': 'utf8mb4',
        },
    }
}

Now you can output and register emoji!

at the end !

We will continue to do our best to add functions, so please registerYoudev

Recommended Posts