I tried to aggregate & compare unit price data by language with Real Gachi by Python

I wanted to compare and aggregate the unit prices of freelance projects by language from before, so I actually investigated it.

Information on freelance projects is aggregated based on the following site Since there was aggregated information on the page of each language, it is based on that

Revatec Freelance https://freelance.levtech.jp

** * Data as of November 21, 2020 **

Cloud.csv

created_at,skill,count,avg_price,max_price,min_price
20201121,OpenShift,5,62,65,60
20201121,Amazon VPC,1,70,70,70
20201121,Google Cloud Platform,55,80,105,53
20201121,Dynamics CRM,1,65,65,65
20201121,Amazon S3,2,80,85,75
20201121,Amazon EC2,2,74,85,64
20201121,AWS,1926,76,135,20
20201121,Google App Engine,7,77,95,60
20201121,Amazon SimpleDB,1,80,80,80
20201121,Force.com,2,70,70,70
20201121,Office 365,9,56,65,40
20201121,Microsoft Azure,141,67,90,50
20201121,OpenStack,21,64,80,45
20201121,Heroku,29,75,102,55

DB.csv

created_at,skill,count,avg_price,max_price,min_price
20201121,Kyoto Tycoon,1,75,75,75
20201121,SQLite,20,72,90,55
20201121,MySQL,1465,75,125,35
20201121,Symfoware Server,4,60,65,55
20201121,Cassandra,14,62,80,45
20201121,PostgreSQL,487,71,135,35
20201121,Riak,5,63,100,45
20201121,Bigtable,3,86,95,80
20201121,SQL Server,375,65,95,35
20201121,Redis,296,78,105,39
20201121,Hbase,5,68,75,60
20201121,Oracle,971,66,115,25
20201121,Sybase,6,75,85,65
20201121,IMS,3,58,64,55
20201121,Access,88,60,145,40
20201121,DB2,79,64,95,35

FrameWork.csv

created_at,skill,count,avg_price,max_price,min_price
20201121,Pyramid,12,77,105,65
20201121,Rails,237,81,115,45
20201121,MyBatis,61,71,85,55
20201121,Node.js,256,77,125,45
20201121,Backbone.js,77,76,115,39
20201121,Knockout.js,4,72,90,55
20201121,AngularJS,152,74,105,47
20201121,Laravel,343,73,125,40
20201121,JUnit,80,73,125,50
20201121,Wicket,3,82,95,70
20201121,Django,89,83,115,55
20201121,Padrino,2,82,85,80
20201121,iBATIS,14,67,80,55
20201121,Silex,4,76,80,75
20201121,FuelPHP,101,76,115,50
20201121,MVC,69,68,95,37
20201121,CodeIgniter,39,74,105,45
20201121,Liferay,1,70,70,70
20201121,React,378,77,115,40
20201121,Spark,19,82,95,55
20201121,PhoneGap,3,86,95,80
20201121,JSF,18,64,75,55
20201121,jQuery,354,71,95,35
20201121,Seasar2,32,70,95,49
20201121,Spring,362,71,115,45
20201121,Bottle,1,75,75,75
20201121,Catalyst,26,64,72,55
20201121,intra-mart,14,70,135,50
20201121,SAStruts,16,71,95,57
20201121,Flask,25,82,125,65
20201121,Sinatra,6,78,85,75
20201121,Struts,129,67,95,45
20201121,Symfony,55,74,95,55
20201121,CakePHP,149,71,95,50

Language.csv

created_at,skill,count,avg_price,max_price,min_price
20201121,PHP,1560,72,145,35
20201121,CSS,650,70,125,35
20201121,C language,269,68,115,35
20201121,UML,7,70,85,50
20201121,C++,374,72,125,45
20201121,C#.NET,147,62,80,32
20201121,Transact-SQL,6,65,75,55
20201121,Kotlin,270,83,125,45
20201121,VB,83,62,85,40
20201121,HTML5,305,72,115,35
20201121,VC,8,61,66,50
20201121,C#,706,68,115,35
20201121,Apex,49,79,135,55
20201121,VBA,165,60,145,30
20201121,ASP.NET,161,65,95,48
20201121,Go,361,81,125,39
20201121,PL/SQL,146,63,85,45
20201121,COBOL,81,61,95,37
20201121,LESS,9,74,115,55
20201121,Swift,407,81,125,50
20201121,VC++,30,65,90,52
20201121,HTML,674,69,125,35
20201121,CSS3,209,75,115,35
20201121,Java,2484,70,145,25
20201121,JSP,54,68,105,55
20201121,Sass,55,71,115,45
20201121,SQL,881,65,145,33
20201121,VBScript,44,62,75,45
20201121,Objective-C,247,76,115,40
20201121,CoffeeScript,16,81,95,65
20201121,JavaScript,1749,72,125,20
20201121,Lua,6,73,80,65
20201121,R language,19,78,95,65
20201121,Smalltalk,2,75,80,70
20201121,Scala,117,83,115,55
20201121,Perl,108,73,110,50
20201121,Shell,193,64,95,45
20201121,SAS,12,61,75,50
20201121,Ruby,652,80,125,39
20201121,ABAP,12,67,80,55
20201121,Python,669,78,145,32
20201121,VB.NET,284,62,85,40

OS.csv

created_at,skill,count,avg_price,max_price,min_price
20201121,Windows,998,63,135,25
20201121,Red Hat,92,66,95,40
20201121,Solaris,37,61,80,40
20201121,AIX,77,63,90,52
20201121,Ubuntu,39,75,90,45
20201121,Unix,236,69,100,40
20201121,AS/400,1,65,65,65
20201121,FreeBSD,3,70,75,62
20201121,HP-UX,21,62,75,55
20201121,Windows Server,387,62,100,33
20201121,Android,563,78,125,35
20201121,Linux,2266,69,135,30

Click here for graph image output source

print_graph.py


import datetime
import os
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

"""
Common variable / constant setting
"""
DATA_PATH = 'data'
now = datetime.datetime.today()
CATEGORIES = {
    'Language',
    'FrameWork',
    'DB',
    'OS',
    'Cloud'
}

"""
Main processing
"""
def main() -> None:

    make_date = now.strftime("%Y%m%d")

    #Directory creation
    os.makedirs(DATA_PATH + '/graphs/' + make_date, exist_ok=True)

    price_graph(make_date)
    count_graph(make_date)
    mix_graph(make_date)


"""
Unit price graph creation
"""
def price_graph(make_date: str) -> None:

    for category_key in CATEGORIES:
        df = pd.read_csv(DATA_PATH + '/levtech/' + category_key + '.csv')
        df = df.sort_values(by=['avg_price'], ascending=False)
        labels = df['skill'].str.replace('[Ah-Hmm-One-龥]+', '', regex=True)  #Delete Japanese
        y_list1 = df['avg_price']

        fig = plt.figure(figsize=(14, 7))

        ax1 = fig.add_subplot(1, 1, 1)
        ax1.bar(labels, y_list1, color='tab:blue')
        ax1.set_ylabel('price')
        ax1.set_title(category_key)

        #Vertical display of X axis
        ax1.set_xticklabels(labels, rotation=90)

        #Add numerical value to bar graph
        for x, y in zip(labels, y_list1):
            ax1.text(x, y, y, ha='center', va='bottom')

        # plt.show()
        fig.savefig(DATA_PATH + '/graphs/' + make_date + '/levtech-' + category_key + '-price.png', bbox_inches='tight', format='png', dpi=300)


"""
Create a graph of the number of cases
"""
def count_graph(make_date: str) -> None:

    for category_key in CATEGORIES:
        df = pd.read_csv(DATA_PATH + '/levtech/' + category_key + '.csv')
        df = df.sort_values(by=['count'], ascending=False)
        labels = df['skill'].str.replace('[Ah-Hmm-One-龥]+', '', regex=True)  #Delete Japanese
        y_list1 = df['count']

        fig = plt.figure(figsize=(14, 7))

        ax1 = fig.add_subplot(1, 1, 1)
        ax1.bar(labels, y_list1, color='tab:orange')
        ax1.set_ylabel('count')
        ax1.set_title(category_key)

        #Vertical display of X axis
        ax1.set_xticklabels(labels, rotation=90)

        #Add numerical value to bar graph
        for x, y in zip(labels, y_list1):
            ax1.text(x, y, y, ha='center', va='bottom', fontsize='small')

        # plt.show()
        fig.savefig(DATA_PATH + '/graphs/' + make_date + '/levtech-' + category_key + '-count.png', bbox_inches='tight', format='png', dpi=300)


"""
Unit price & number of projects graph creation
"""
def mix_graph(make_date: str) -> None:

    for category_key in CATEGORIES:
        df = pd.read_csv(DATA_PATH + '/levtech/' + category_key + '.csv')
        df = df.sort_values(by=['avg_price'], ascending=False)

        labels = df['skill'].str.replace('[Ah-Hmm-One-龥]+', '', regex=True)  #Delete Japanese
        y_list1 = df['avg_price']
        y_list2 = df['count']

        fig = plt.figure(figsize=(14, 7))

        left = np.arange(len(labels))
        width = 0.3

        ax1 = fig.add_subplot(1, 1, 1)
        ax1.bar(labels, y_list1, width=width, color='tab:blue', label='price')
        ax1.set_ylabel('price')
        ax1.set_title(category_key)

        ax2 = ax1.twinx()
        ax2.bar(left + width, y_list2, width=width, color='tab:orange', label='count')
        ax2.set_ylabel('count')

        #Vertical display of X axis
        ax1.set_xticklabels(labels, rotation=90)

        #Add numerical value to bar graph
        for x, y in zip(labels, y_list1):
            ax1.text(x, y, y, ha='center', va='bottom')

        for x, y in zip(left + width, y_list2):
            ax2.text(x, y, y, ha='center', va='bottom', fontsize='small')

        #Output legend
        handler1, label1 = ax1.get_legend_handles_labels()
        handler2, label2 = ax2.get_legend_handles_labels()
        ax1.legend(handler1 + handler2, label1 + label2)

        # plt.show()
        fig.savefig(DATA_PATH + '/graphs/' + make_date + '/levtech-' + category_key + '-mix.png', bbox_inches='tight', format='png', dpi=300)


if __name__ == '__main__':
    main()

Actual graph

--Price is the average unit price (10,000 yen) --count is the number of cases ――The graph in which the unit price and the number of projects are mixed is in descending order of unit price.

language

By unit price

By number of cases

Unit price and number of projects mix

The top is Kotlin. I'm convinced that I hear that there is a shortage of Android engineers now. If you look at the number of projects and the unit price for the web system, Ruby, Python, and go seem to be good for cospa. I had the impression that Java had a high unit price for the time being, but it is surprising that PHP is more expensive ... Is the demand too high?

Then the framework

By unit price

By number of cases

Unit price and number of projects mix

The top is PhoneGap. I heard it for the first time, but it seems to be a cross-platform mobile application development framework developed and released by Nitobi. If it's a web system, Django and Rails seem to be good at cospa. Since React is used for web and application development, the number of projects and unit price are good. Vue had no information at the site source. ..

By unit price

By number of cases

Unit price and number of projects mix

The top is Bigtable. I often hear Bigquery, but the difference seems to be like a large DB that runs faster than Bigquery. Regarding DB, there is no doubt if you remember MySQL in terms of cost performance. Oracle and SQL Server are surprisingly low. why,,,

Cloud

By unit price

By number of cases

Unit price and number of projects mix

The top unit price is GCP! AWS is overwhelming when it comes to the number of projects! For the time being, it seems better to remember AWS at least. If the demand becomes too high, the unit price tends to decrease, so considering the future, it may be better to remember GCP as well. Azure has a delicate number of deals and unit prices

Impressions

The result was quite interesting! If you do web-based things from now on, Python may grow. It's easy to build an environment by actually touching it. Rich library. I was particularly impressed with how easy it is to create CSV → graphs! It's really nice to be able to quickly create and image with the back end rather than using the html or js library to go crazy ~

This time, it is the result of investigating one of the project sites, so please use it as a reference! Information on Vue, Graphql, and Rust wasn't listed, so I'm worried about that.