[PYTHON] It's been a year since I was self-taught and moved to the web application development department, so I look back

Introduction

Just in January of this year, I was transferred to the development team, We are about to enter the year. It may be the beginning of the year, so I will describe what I want to do for this year and a review.

About the product we are developing

The development team I moved to I am in charge of the access log measurement service for online behavior data. There are many ties with the digital advertising department, etc. The platform is mainly AWS, most of the sources are PHP, and some are Python / Java.

My condition after being transferred

Originally I was interested in programming, I was doing a lot of things between work. I wanted to make it myself I want to spend a little more time studying, so I started going to something like a programming school.

Learn from the basic grammar of Ruby, such as MVC model, I created a Twitter-like service using Ruby on rails in a form close to copying. I graduated because I felt the limit to my growth at the programming school. After that, without making an original application, I was busy with work and spending my days in a hurry.

In terms of business, programming schools give top priority to creating "things that take shape". (Otherwise everyone will be frustrated) So, I don't really understand the meaning, but if you follow the teaching materials, it will work. Since it is done in a cloud environment, there is not much difference between environments, The feature is that if you follow the teaching materials, there will be almost no errors.

I think it was a good start, but I now think that I didn't have much knowledge about infrastructure such as around servers.

While studying in such an environment, I moved because of a relationship, At that time, I felt like the following. ・ I understand HTML / CSS / ruby / python / mysql to some extent ・ MVC model knows the outline -Knowledge around the server is almost 0. vi When I open it, I can't close it and I'm impatient ・ Of course, no knowledge of networks, software, or hardware ・ AWS has only touched Cloud9 and EC2

From here, I think there is much more that can be done, I want to speed up my growth next year, so I will look back on what I did, what I was able to do, and what I wanted to do next year.

What I did this year

Looking back on this year, I think it can be divided into three stages: training period, division inquiry response period, and development work period.

Training period (January-April)

Immediately after being assigned Because it was a rare pattern of transfer from the business division side to development, I have almost no work. Get the theme of your research, write Hello world in various languages, I was doing file output. We also focused on studying HTTP protocols and networks during this period. In addition, basic information processing was also acquired during this period.

Inquiry response period (May-October)

Because I got a good overview of our services Began responding to inquiries, The work of bringing in specific information from the DB has increased considerably. Also, some are affected by Chrome's Samesite attribute, I was also involved in the response.

To development work (July-)

I think I started to touch the development work from around July.

・ Validation implementation in JS ·Dashboard ・ Intercept correction of machine learning model ・ Correct data linkage with related services ・ Service release I was allowed to do it. All of them were like touching for the first time, so Although it was painful, I think it was a great learning experience.

What I can do and remember this year

Training period (January-April)


What I learned
・ Basic knowledge enough to receive basic information processing ・ Grammar of each program (HTML & CSS, Python, PHP, Javascript, jQuery, SQL) that can be done with Progate ・ How to use Git ・ How to use Redmine -Classification of Linux distributions -Package manager classification


What I can do
・ Create a simple desktop application with python ・ Easy operation of Excel with python

There was nothing I could do during this time, so I was just inputting. If I didn't have any goals, it would be hard for me to do this, so I input with the goal of passing basic information processing, and I was able to pass it safely. Being able to pass the exam gave me some knowledge, which gave me confidence.

I think it was good to have a way to approach things that I don't understand around here. For example, try inputting from a different perspective, Take a moment to input other things and then try to learn again. The rest is the importance of output. In particular, I tend to get a rough idea of things. In my own interpretation, I just created an image, I noticed that it was not well verbalized.

After all, if you can't verbalize it, you can't use it for communication with people, so I think it was an important notice.

Also, with the knowledge of Linux distributions, Information gathering around environment construction has become smoother. Previously, I saw an article such as "How to install XX on Ubuntu" There were many cases where I copied the code in that article to CentOS and it didn't work, but ... It has become possible to classify information that can be applied and information that cannot be applied to some extent.

Inquiry response period (May-October)


What I learned
・ Infrastructure configuration of in-house service ・ AWS knowledge such as EC2, ELB, Opsworks, CloudFront, CloudWatch, Athena, S3, etc. ・ Basic knowledge such as folder structure of CakePHP and Laravel ・ Knowledge about cookies and cookie attributes (Samesite, domain, etc.) ・ Load test method using Jmeter


What I can do
・ A set of Linux commands -Program creation with Shell script (file input / output, DB operation, replacement, etc.) ・ AWS CLI (s3, athena) ・ Make the access log into the desired shape with Athena ・ Make the access log into the desired shape with php and python -Extracting data from multiple tables with mysql (I was not good at join) ・ Perform a load test with Jmeter and output a little less than 1000 imps per second.

In the period around here There were many responses to inquiries such as issuing such data from the business division and how many such requests were made. Opportunities such as data extraction from the access log accumulated in s3 have increased significantly, but Do similar extractions through PHP, try with Pandas, do everything with Athena, etc. I was able to challenge in various ways. There is a decent amount of data, such as character string extraction and flagging conversion, etc. I think I was able to study because I was able to do various things other than that. Regarding mysql, I learned the sensuous part such as whether joining in this order makes it heavier.

Also, thanks to the fact that I wrote in Shell that this work is troublesome. A lot of programs in Shell are now available.

To development work (July-)


What I learned
-How to deploy each server of your company ・ The touching part of Vue.js ・ Overview of Java (object-oriented) -Easy usage of Maven, Ant build tools ・ Concept of logistic regression analysis ・ AWS knowledge of ECS, ECR, CodeCommit, AWS Batch, Kinesis, KCL, etc. ・ Knowledge of Fluentd


What I can do
・ DOM manipulation with JS ・ DOM manipulation with Jquery, Ajax -Simple Vue.js fix · Use of css framework ・ Auto scaling using ECS and ECR -Manipulating files of several GB using python libraries such as pandas and numpy ・ Customize around Auth, send email, create validation rules with Laravel ・ (Accuracy aside) Development estimate -Java copy-based modification, compilation, and execution

Since July, I have gradually been able to receive development tasks. Since I often did it with functions that I made for the first time or library frameworks that I touched for the first time, I am very grateful that I was able to broaden my horizons.

If anything, there are many opportunities to touch the front side, It was my first time to touch JS properly, In the second half, it was good to touch the world of Vue.js lightly.

Also, because I was allowed to carry out the release work, Specific knowledge of service architecture I was able to organize the knowledge around ECS and ECR.

Other than that, I soft-touched Java, I was able to alleviate the difficult image of Java to some extent. (I hated Oracle DB)

What I want to do next year

As for technology, it's not something I want to do, but While actively collecting information I want to be able to output instead of input when I have the opportunity. Also, since most of the level feelings are still touching the introduction of each technology, I would like to emphasize stepping up from beginners to intermediates.

API There has been no touch on the API so far. I've done something like hitting Tabelog and Rakuten's API, I have no experience of implementing the API side, so I want to do it. I think it's from the Restful API input first, Isn't it like trying to publish a simple API? ..

Laravel While being exposed to various languages, This is ok! I'm a little worried that there is nothing I can think of. Currently Laravel is decent to some extent, With Laravel, I want to bring it to a state where I can understand everything.

Software architecture

Join the PHP conference earlier this month Increased interest in architecture. Because I borrowed Design It! What kind of thinking should I take on a daily basis? I would like to build such a part. The architecture is wide, so First of all, I would like to dig deeper into the fields that I am interested in from where I am interested.

Test code

The most painful part of my development work for a year was manual testing. I've only done almost enough to run human-written test code, so I want to be ready to write test code quickly.

In my career, when I moved to the development department in the liberal arts, Proof of some knowledge by qualification seems important. So, while aiming to acquire knowledge to the last I would like to obtain the following qualifications.

Passed applied information processing

From security to hardware, software, DB, etc. I would like to acquire it because it will be an opportunity to study a wide range of fields. I got basic information processing, but because it was a selection formula, It can be said that it has become established as knowledge. I will gradually start learning from the beginning of the year and acquire it in April.

Acquisition of AWS Solution Architect

I've been touching AWS a little bit in my business, I want to keep the basics of services that I have never touched, so I will acquire them. I will take it after applied information processing, so The goal is around June.

Gain more security knowledge (Tokumaru test?)

PHP Engineer Certification Corporation for PHP

There are quite a few parts where security is a concern and the speed of creating services is slowing down. I also want to gain confidence here, I definitely want to receive it. It is said that a beta test will be conducted by early summer next year, so I don't know what will happen, but I'll keep an eye on it.

Create (complete) your own service

I was making a web application for a while, but I've been sloppy, so I want to get rid of it as soon as possible. Rather than sticking to the details and not deploying easily Try to put it out for the time being when it is completed to some extent, I will try to correct it while watching the reaction.

Summary

It's been a year since I moved I learned more than I expected. I don't think I could have gained this knowledge unless I was transferred. However, I'm still a chick, so I will do my best next year so that I can grow even more.

Recommended Posts

It's been a year since I was self-taught and moved to the web application development department, so I look back
When I tried the AtCoder Beginner Contest, it was a terrible result, so I look back
Note that I was addicted to accessing the DB with Python's mysql.connector using a web application.
The road to web application development is a long way off
I want to make a web application using React and Python flask
What I was addicted to when creating a web application in a windows environment
[ES Lab] I tried to develop a WEB application with Python and Flask ②
I made a program to look up words on the window (previous development)
I set up TensowFlow and was addicted to it, so make a note
I made a tool to automatically generate a state transition diagram that can be used for both web development and application development
I was surprised to receive a nice review when I wrote Python to CheckIO and its explanation
I wrote AWS Lambda, and I was a little addicted to the default value of Python arguments
Since the Excel date read by pandas.read_excel was a serial number, I converted it to datetime.datetime
I tried to find out the difference between A + = B and A = A + B in Python, so make a note
A story about everything from data collection to AI development and Web application release in Python (3. AI development)
GCP's Cloud Shell Editor was clogged up with the Python development environment, so I managed to rebuild it in a vacant lot.