Easy code review to get started with Jenkins / SonarQube: Static analysis

Introduction

In this article, we will perform static analysis [^ 2] of the code within the development (programming) phase [^ 1] performed by multiple people in the group, and help to eliminate the problems contained in the newly developed part at an early stage. I will show you how.

[^ 1]: [Waterfall model] using Java (https://ja.wikipedia.org/wiki/%E3%82%A6%E3%82%A9%E3%83%BC%E3%82 % BF% E3% 83% BC% E3% 83% 95% E3% 82% A9% E3% 83% BC% E3% 83% AB% E3% 83% BB% E3% 83% A2% E3% 83% 87 % E3% 83% AB) I wrote it assuming development, but I think that it can be applied not only to the waterfall model, and SonarQube itself can support languages other than Java.

[^ 2]: A type of computer software analysis method that analyzes without executing an executable file. ([Wikipedia](https://en.wikipedia.org/wiki/%E9%9D%99%E7%9A%84%E3%82%B3%E3%83%BC%E3%83%89%E8% From A7% A3% E6% 9E% 90))

Premise

――It is assumed that the development flow is basically "Create a development branch from the master branch and send a pull request from that branch to the develop branch when development is completed". --Assuming a situation where the number of reviewers is not large compared to reviewee.

problem

--Reduce the burden on reviewers --Have you ever pointed out the same coding rule violation many times (how to use logs, omission of annotations, etc.)? -When you search with SearchBucket, do you find many similar points with the same keyword? --Preventing the release of unintended defects --Memory leak, NullPointerException, etc. --Targeting newly developed parts ――I would like to put the existing problem on hold.

SonarQube Aiming to solve the above problem, this time we will try to introduce and utilize SonarQube. SonarQube is, in a nutshell, an "OSS for continuously measuring and managing code quality."

--If there is a part that violates the specified coding rule, it will be identified and presented (along with the solution policy). ――It manages such violations as "technical debt" and records the transition in chronological order.

See the Official Documentation (https://docs.sonarqube.org/display/SONAR/Documentation) for more information.

SonarQube® software (previously called Sonar) is an open source quality management platform, dedicated to continuously analyze and measure technical quality, from project portfolio to method.

Install SonarQube Server

First, build a server and prepare a place to store code quality measurement results. Detailed procedure is Official document and [This document](http://blog.applibot.co.jp/blog/ 2017/09/14 / static-code-analysis-with-sonarqube /) is detailed.

Static analysis and result confirmation with SonarQube Scanner

Once you've built your server, it's time to statically analyze your code.

Static analysis: Maven

After setting according to Analyzing with SonarQube Scanner for Maven, in the directory with pom.xml

mvn sonar:sonar

Will perform a static analysis.

Check results: On the web console

The result is

sonar.host.url

You can see it as such a screen under the URL set in. (The image doesn't seem to be for Java code, but for reference.)

Source: https://stackoverflow.com/questions/36940993/vs-unit-test-results-file-trx-is-not-displayed-into-sonarqube

The problems found and their solutions are presented, for example:

Source: https://docs.sonarqube.org/display/SONARQUBE52/Technical+Debt

Check results: In the IDE

If the analysis result is registered on SonarQube Server, it can be confirmed on the IDE. To do this, depending on your IDE, SonarLint or [radar-netbeans](http://plugins.netbeans.org/plugin/51532/radar- netbeans) is installed. For example, if you install radar-netbeans on NetBeans, the problem ("Issues") is pointed out like this. You can make corrections smoothly by going back and forth between the relevant part of the code and the detailed description of the problem.

Source: http://plugins.netbeans.org/plugin/51532/radar-netbeans

merit and demerit

Here are the advantages and disadvantages of actually introducing SonarQube.

• merit --You can quickly find (and fix) problems during development on the web console or IDE.
• Demerit --SonarQube Scanner needs to be installed and configured by each engineer. --If you want to ignore the problem pointed out, you can ignore it.

Combine with Jenkins

SonarQube Scanner needs to be installed and configured by each engineer.

In order to reduce the disadvantage, let's think about creating a job that can be easily executed from Jenkins that is used in common by the group.

script

For each development branch, create a script (analyze.sh) for static analysis of SonarQube as follows.

--Analyze for the master branch (if you haven't done so already) --Analyze the development branch --sonar.projectKey is unique for each development branch --Sonar.projectVersion changes between master branch analysis and development branch analysis

analyze.Overview of sh

#!/bin/sh -xe

(Omission)

SONARQUBE_PROJECT_KEY_FROM_API=`curl -s -X GET "${SONARQUBE_HOST_URL}/api/components/show?key=${SONARQUBE_PROJECT_KEY}" | python [Path to file]/json_value.py component key`

# clone repository
git clone ssh://git@[Bitbucket SSH URL with port]/${BITBUCKET_PROJECT_NAME}/${REPOSITORY_NAME}.git
cd ${REPOSITORY_NAME}; pwd

# if key not exists in SonarQube server
if [ -z ${SONARQUBE_PROJECT_KEY_FROM_API} ]; then
    # (a) mvn clean package (baseline branch)
    git checkout ${GIT_BRANCH_BASELINE}
    ${MAVEN_COMMAND} clean package -Dmaven.test.skip=true

    # (b) run sonar:sonar to ${SONARQUBE_PROJECT_NAME} (baseline branch)
    ${MAVEN_COMMAND} sonar:sonar -Dsonar.projectKey=${SONARQUBE_PROJECT_KEY} -Dsonar.projectName=${SONARQUBE_PROJECT_NAME} -Dsonar.projectVersion=${SONAR_PROJECT_VERSION_BASELINE} -Dsonar.sources=src/main
fi

# (c) mvn clean package (developing branch)
git checkout ${GIT_BRANCH}
${MAVEN_COMMAND} clean package -Dmaven.test.skip=true

# (d) run sonar:sonar to ${SONARQUBE_PROJECT_NAME} (developing branch)
${MAVEN_COMMAND} sonar:sonar -Dsonar.projectKey=${SONARQUBE_PROJECT_KEY} -Dsonar.projectName=${SONARQUBE_PROJECT_NAME} -Dsonar.projectVersion=${SONAR_PROJECT_VERSION} -Dsonar.sources=src/main

json_value.py

# encoding: utf-8
import json
import sys

# Get JSON from stdin
dic = json.loads(raw_input())

# get keys from arguents
args = sys.argv
# args[0] is this program name
args.pop(0)

for key in args:
    if isinstance(dic , list): 
        # if current value is list, change key to int
        key=int(key)
    try:
        # set dic[key] as next value (or result)
        dic=dic[key]
    except: 
        # If failed, print empty string and exit. (might be on KeyError and IndexError)
        print ''
        sys.exit()

# print result
print dic

Jenkins job settings

Create a Freestyle job and check "This project is parameterized" so that you can specify the required parameters for your script.

Jenkins job execution

Running the job creates a dashboard on the web console that corresponds to your development branch (if you don't already have one). Problems detected in the difference between the master branch and the development branch (newly developed part) will appear in "New Issues". You can see the details of the problem by clicking on the area circled in red.

merit and demerit

• merit --You can find the problem to be fixed just by specifying some items such as the branch name under development.
• Demerit --If you want to ignore the problem pointed out, you can ignore it.

Summary and future issues

With the aim of "reducing the burden on the reviewer" and "preventing the release of unintended defects", Jenkins / SonarQube can be used to provide a useful method to quickly eliminate problems contained in newly developed parts. I introduced it.

On the other hand, we also found such issues.

If you want to ignore the problem pointed out, you can ignore it.

I think this is largely due to the circumstances of the group rather than Jenkins or SonarQube. After talking to the members who actually used the Jenkins job, I think the following points are likely to be a bottleneck.

--The importance of static analysis is not well understood and shared ――There were no major problems in the manual reviews so far (or even if problems occur, we have been able to deal with them). --SonarQube's default rule-based suggestions don't always fit the group's current situation. ―― “The burden of reviewee (not reviewer)” is increasing. --I can't remember this Jenkins job during development (too much else to do). ――Even if you remember, it is difficult to actively fix it due to problems such as schedule, skills, and consistency with existing code.

In the next related article,

--SonarQube rule selection

• Automation

I would like to share either of them if there is a prospect of improvement.

Related articles

-Can I use Jenkins to automatically test GitHub Pull Requests? ――I don't use SonarQube, but I have the same purpose of "reducing the burden on the reviewer" and "reducing the possibility of mistakes getting into the release". -How to statically analyze test code with SonarQube? --I examined how to set the analysis target of SonarQube Scanner.