[PYTHON] What I have learned in the past year as an elderly person (unsuitable for my age)

Overview

Write what you are learning every day as an elderly person. I am inspired by the younger generation who are enthusiastic about learning skills. The front / web system is too ant to keep up with learning, so I specialize in my area of interest.

By the way, there is no organizational theory or anything like that.

Self-introduction

――Soon 50 years old (Although it looks like "Is that so?", I can't beat the advancing presbyopia) ――In the old days, the foundation, now for some reason, the business team, all but the business core. --Some time ago LPIC, Java, Oracle --Recently, statistics and Python are the main focus (however, it has nothing to do with business) ――We are planning to work on a data analysis competition soon (let's do it together)

We repeatedly propose statistical tests and in-house education (learning support) for Python. I hope you will pick it up someday. http://www.toukei-kentei.jp/ https://www.pythonic-exam.com/exam

Looking back on the past year

The theme is machine learning and deep learning, so I want to be able to create something by myself, not just using something. I think about what I need and I think there are four big ones.

Since I am mainly self-taught, I tend to be satisfied with reading and implementing it, but most of the time I still lack understanding. At the same time, I try to take a qualification test to improve my training. (If you have learning support from the company, you can use the recommended method, but this field is "self-sufficiency".)

1. Improving basic skills based on mathematics

I also did mathematics as a hobby, so there is no problem with "differentiation, partial differentiation, algebra" required for machine learning and deep learning. I remember avoiding mathematical statistics because it was "unpleasant and unpleasant," but I learned it again by myself. A few years ago, I obtained the qualification of Statistical Test Level 2 and Statistical Surveyor. Currently, I am studying for the first grade of statistical test. http://www.toukei-kentei.jp/

Exercises are important, so I not only read them, but also move my hands to solve the problems. It's impossible when commuting, so before starting work or at home. Right now I am solving without setting a time, but considering "practice", I want to be able to solve a huge amount in a short time. As you can see by taking the exam, there is not enough time at all, that is, people who are doing their main business are doing this amount. .. ..

This is recommended for those who want to learn statistics from now on. ** Introduction to self-study statistics ** https://www.amazon.co.jp/dp/4535787204

2. Python implementation power

The theme is understanding language specifications and implementation amount.

There weren't many cases in the company, so I taught myself. It is not enough just to read it somehow, so I learned the basic skills by copying sutras and qualification test. However, it is not a substitute for being able to freely analyze data once it reaches the pass level of this qualification test, and it is still insufficient.

** Python 3 Engineer Certification Basic Exam ** https://www.pythonic-exam.com/exam/basic ** Python 3 Engineer Certification Data Analysis Exam ** https://www.pythonic-exam.com/exam/analyist (Obtained when the beta test was held, this test will be held next year)

After reaching this level, we will repeat copying sutras and solving problems with the themes of machine learning and deep learning.

In addition, I am referencing the websites of Gachi. At this stage, you can feel the gratitude of learning from "Hajipata" and "Basics of Modern Mathematical Statistics".

3. Algorithm devise

I wrote and moved, but I can't say it's OK. Efficient algorithm training is required even in machine learning, which requires a large amount of calculation, and it is also necessary to increase the number of algorithms that can be understood and used. Of course, we need theory and inspiration.

** Competitive programming contest site "At Coder" ** https://atcoder.jp/?lang=ja Held every Saturday, there is a problem statement print page, so I carry the output and train it using a little gap time. .. ..

4. Business + α

Shortly before I was transferred to the business team, "automatic maintenance of infrastructure" and "cloud use" became a hot topic, and it was a time when cost reduction and error reduction by not using people were advocated. It was around the time I started learning because I thought it was true.

Then, did the business team have more penetration of initiatives such as CI than the basic team? .. .. There was no big difference ... Of course there was CI by maven + jenkins, but it was up to partial automation.

So when I started automating, it was difficult to return to manual management using Excel and the management ledger, so various management and manual work were automatically executed using Python and Jenkins (scheduled in the middle of the night and in the morning). Is all done).

I think I was able to improve Python in practice in the process of this improvement task. There is a sense of accomplishment when Jenkins can do all the lazy tasks in the near future. Looking back, I wrote quite a few small tools.

• pandas
• numpy --Graphing --Data editing --Handling of Excel in python --Linking with python and Redmine --Cooperation with OS --Cooperation with Postgres --Problem handling know-how that is easy to trip --Text editing cooperation with bash (awk / sed / find / grep)
• Regular expressions

And "de-Excel macro" has also been realized. I swear I will never write a macro again. (Macro occupies the terminal with high load and long time macro unless a person starts it ... I can't go back anymore)

By going from manual work to Jenkins, I think that I can spend more time and have the spare capacity to do plus α things, which leads to a good cycle. Recently, when I discover some problems in development and operation, I feel that I usually make countermeasures on the same day and start operation without human intervention. Since I sometimes make it by rushing, there is also a rough code, and I will do refactoring soon.

** You have made "your own RPA" with Python and Jenkins. .. .. ** **

Since this is an external site, there may be a problem if you write down the internal circumstances in detail, so please contact us individually if you are interested.

from now on

It's still halfway through, but I think we're finally ready to take on the data competition. It's been a year of detours, but I'm hoping that the basic skills that I've honed somewhere will come to life.

next year I would like to expand what I can do by participating in data analysis competitions and Gachi seminars. If you have someone who has specialized in studying when you were a student, let's do it together.

https://www.kaggle.com/ https://signate.jp/competitions

** Recent interests **

――Every day I feel that I can't easily talk about the relationship between things without causal reasoning. ――I'm in the business team, but I want more specific analysis themes. .. .. Then I decided to learn from Saber Metrics as a hobby. There seems to be a hint for creating a feature index. ――I switched my home environment from notebook to vscode. Make progress (notebook default support). By the way, it is vimer in the company (use it without a plug-in). --Bicycle, brevet (200km, 400km) participation