<!-TODO: Be careful if the various links and notations are for 2018. Pay particular attention to the twitter link! !! -> The presentation material of the title session of JJUG CCC 2019 Spring has been released on Qiita (twitter: # ccc_a2 //twitter.com/search? q =% 23ccc_a2)).
<!-TODO: Replace with screenshot image of Speakers deck-> <!-TODO: Link destination required correction! !! ->
The content is almost the same as the above slide, so I hope you can see it in an easy-to-read manner.
<!-TODO: Keep the back of the time series->
Our company M3 Co., Ltd. was founded in 2000 (J2SE
, the era around Java 2).
I've been using Java since the days when Maven, Gradle, and Ivy weren't there yet.
When you do git clone
, the following file will be ...
lib/app/hibernate-validator-4.3.0.Final.jar
lib/app/aws-java-sdk-core-1.10.27.jar
lib/app/aws-java-sdk-s3-1.10.27.jar
lib/app/generator-runtime-0.3.0-20050901.1909.jar
lib/app/xalan.jar
lib/app/crimson.jar
...Continues endlessly...
As a result of the increasing number of libraries used over history, 182 .jar
s were rolling.
.jar
I was in a state of maintenance with such a process:
Example: I want to upgrade the version of ʻaws-java-sdk` to use new features
aws-java-sdk system indirectly depends on multiple libraries
I don't know how to upgrade 182 .jar
s
Give up on using new AWS features: cry:
I'm not using a jar, so I want to delete it
Maybe one of the jars depends on it ...
Unnecessary jar cannot be cleaned
To hinder JVM version upgrades and vulnerability countermeasures: cry:
I want to check vulnerability information
Check 182 .jar
s with vulnerability information on a regular basis
Tough & limited
I tend to leave it alone
Gradle, maven, etc. can be automated with OWASP dependency-check
plugin ...
Other than this system, maven and Gradle have already been installed, but ...
--The system has been around since its inception and has a particularly long history. --Ant's build script is tightly assembled, and batch replacement is difficult ――Since the usual development work goes around without problems ... (* As long as the darkness around the library is ignored) </ small>
Under these circumstances, there may be cases in the world where package management tools are not included.
But how do you implement it in an existing project ...?
――What do you do about the points that tend to increase the work load? ――What are the side effects and effects of adding it?
→ Go to the next chapter
<!-==================================== # Section: Strategy Part (Gradle) == ==================================->
~ Historic system improvements __Possible __scope definition ~
Retrofitting into existing systems presents a number of challenges:
――Is it good that the jar changes before and after the introduction of the tool? --What to do with jars that are difficult to re-obtain or that do not match central? --Would you like to modify the build process?
It's better to have a policy so you don't get swayed by challenges.
--Do not make a difference in the .jar
file before and after the introduction of the package management tool.
--To reduce concerns about the impact on the app and the burden of testing
--Make sure that the only unavoidable difference is a harmless difference (method will be described later) </ small>
--Inventory and ver up are the scope of the next improvement
--Do not touch the build process
--First, improve package management!
Scoped to benefit from package management tools, Created a situation where you can take on small and meaningful challenges.
.jar
to Git: + 1: Benefits:
--Easy to migrate from Ant etc. --Always use the same jar file
: -1: Disadvantages:
--Ant remains
--If you really have a problem with Ant, replace it at that time
--Repositories will grow with repeated updates of .jar
--Sometimes it will be separated as Git submodule
Identify the true trouble
and merit to change
Consider the necessary and sufficient minimum scope
for that.
Achieve the goal with small challenge
(high cost performance, small damage even if you fail) </ small>
It ’s not “Legacy, so you have to kill everything !!!” Improvements while following the existing good points and points that do not need to be changed.
~ Specific measures to deal with various issues that tend to be retrofitted ~
Gradle or Maven
Gradle is recommended for applications like this one:
--Flexibility: Flexible processing can be described in Groovy and Kotlin DSL --Maven may not be able to handle the composition of historic projects --Investigative: Groovy Javadoc is easier to read (subjective) </ small> --Productivity: Take full advantage of the IDE's debugger --You don't have to fight like a black box --Reproducibility: Gradle wrapper makes it (relatively) easy to ensure behavioral reproducibility
* In general, Maven is better in some cases </ small>
In order to introduce Gradle while utilizing the existing build system I want to copy the following files to the specified directory with Gradle:
--Jar that depends on compile time and execution time --Jar that depends on unit test
I also want to automatically maintain the .classpath
file (described later) </ small>.
Surprisingly easy:
def compileJarCopyTo = 'development/lib/app'
task downloadJars() {
doLast {
//At the copy destination*.Erase the jar
delete fileTree(dir: compileJarCopyTo, include: '*.jar')
//make a copy
copy {
from configurations.compile
into compileJarCopyTo
}
}
}
copy
task
Convenient to specify the jar to be copied and include, exclude, etc .:
copy {
// testCompile =Copy the dependent jars in test
from configurations.testCompile
into testCompileJarCopyTo
//Ignore jars that match the specified pattern
exclude 'jmockit-*.jar'
}
To copy the Source jar, write code to manipulate these goggles:
import org.gradle.jvm.JvmLibrary
import org.gradle.language.base.artifact.SourcesArtifact
import org.gradle.api.artifacts.query.ArtifactResolutionQuery
import org.gradle.internal.component.external.model.DefaultModuleComponentIdentifier
(ʻorg.gradle.internal` has also appeared ...) </ small>
→ Refer to the code for Gradle 2.x of the predecessor, and refer to the Gradle 5.x compatible version
Copy source jar files into directory (Qiita ) Posted in
.classpath
fileA configuration file that allows the IDE to load jars and source code. Originally for Eclipse, it also supports InteliJ IDEA.
I want to maintain this file with Gradle because it's a big deal.
It can be generated just by applying Gradle's ʻeclipse` plugin ...
.classpath
Absolute path problemThe path in .classpath
generated by Plugin is an absolute path.
It is inconvenient because it becomes an environment-dependent path and cannot be shared with Git.
solution:
--Create .classpath
each time by gradle eclipseClasspath
at your own hands
-: -1: It takes time for everyone to do it every time
-: -1: The reproducibility of the build decreases (even though it worked at hand! Phenomenon)
--Generate .classpath
in relative path notation with Gradle and commit with jar file
-: + 1: Just execute it together with a copy of the Jar file etc. OK
.classpath
in relative path notationʻEclipseplugin's
pathVariables'.': Instead of projectDir`, if you extend it using the plugin hook as shown in Reference site good:
eclipse.classpath.file {
whenMerged { classpath ->
classpath.entries.findAll { entry -> entry.kind == 'lib' }.each {
it.path = it.path.replaceFirst("${projectDir.absolutePath}/", '')
it.exported = false
}
}
}
You can freely control the contents of the .classpath
file with whenMerged
For example:
--Rewrite the reference path of the jar file with arbitrary logic
--In our case, only some files are placed in different directories.
-- classpath.entries.sort
controls the sort order of .classpath
--Useful for load-order-dependent library measures such as JMockit
--classpath.getEntries (). removeAll {return (conditional expression)}
--You can exclude jars that you don't want the IDE to load
Manual management jar filenames and Gradle-derived jars often do not match. For example:
--Manually managed file name: crimson.jar
--Gradle download result: crimson-1.1.3.jar
There is a problem when referencing with a fixed file name. In particular, there are the following:
--A fixed file name in the classpath
of build tools such as Ant
--File name fixed in the classpath
option of the JVM when the program starts
*
wildcards in Ant, JVM classpath
(> = 6), shell, etc.
--Note that the interpretation of *
is slightly different.The latter is a hassle, but it is a temporary effort, so it is better to deal with it in the long run.
<!-==================================== # Section: Strategy Part (Jar Management) = ===================================->
~ Techniques for handling jar files that came from beyond history ~
<! ---- The jar contained in the all-in-one jar is squishy->
--I was able to get the jar again, but the binaries do not match
--SNAPSHOT
version
--Although it is non-SNAPSHOT
, it has been replaced and released.
--Different binaries are placed between Maven repositories
--Original patch of unknown background: imp:
--The official page has disappeared and you cannot re-obtain the jar or source
Don't leave the discrepancy, just check the specific differences:
--You can limit the scope and risk of introducing a package management tool. --The range to be tested and the test method will be clarified.
I issued a SHA-1 hash of the jar file before and after Gradle conversion, We examined the specific differences for jars whose hashes do not match.
The jar file is a zip and when unzipped you will see the following files:
.class
file (Java bytecode)META-INF
Except for .class
, most text comparison tools can be used for comparison.
Example when there is a difference in META-INF / MANIFEST.MF
of lucene-analyzers-2.4.0
: </ small>
8c8
< Implementation-Version: 2.4.0 701827 - 2008-10-05 16:46:27
> Implementation-Version: 2.4.0 701827 - 2008-10-05 16:44:47
The .class
file is a Java bytecode binary.
By converting to text representation with javap -c
included in the JDK
It becomes possible to compare with the text comparison tool.
Example when there is a difference in guice-servlet-1.0.jar
:
2c2
< final class com.google.inject.servlet.ServletScopes$1 implements com.google.inject.Scope {
> class com.google.inject.servlet.ServletScopes$1 implements com.google.inject.Scope {
(replaced by jar with final
added) </ small>
Measures other than suspension of use of the library:
-(: imp: Deprecated) Upload to public repositories such as Central
-: -1: License or morally unfavorable
-: -1: To publish an unconfirmed jar that matches a known jar
-(: thinking: subtle) Use the .java
file of the target library directly
-: -1: Since the source is used directly, original modifications and tight coupling tend to occur.
-: -1: There is a problem depending on the license
-(Recommended) Managed by private repository for internal use
-: + 1: Rights problems are less likely to occur
-: + 1: Easy to grasp and delete the usage status of the library
Not limited to non-retrievable jars, but also for managing in-house libraries, etc. It is convenient to have a private repository.
Common construction method:
file: //
--Needs synchronization between machines
--Tends to compromise build reproducibility
--There is also a risk of losing the jar and the trouble of backing up.With Gradle, you can do this simply by:
repositories {
maven {
url 'http://central.maven.org/maven2/'
} // mavenCentral()I do not dare to use the abbreviated notation(See below)
maven {
url "http://URL of a private repository"
}
}
<!-TODO: This description may not be necessary depending on the reader-level assumptions ... But should I write something like Gradle for people who don't know maven? ->
Write the metadata in Maven XML (pom.xml
) format and upload it with the jar.
It may seem awkward at first glance, but it's easy if you focus on this purpose:
<project>
<modelVersion>4.0.0</modelVersion>
<!--Identification information for this jar itself-->
<groupId>com.m3</groupId> <artifactId>m3generator</artifactId>
<version>1.0.0</version>
<!--Information on the jars this jar depends on-->
<dependencies>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
...
It looks convenient because Artifactory automatically generates a POM ...
<dependencies>
is often empty or incorrect
I can't get information about the dependent libraries of the uploaded jar
The benefits of dependency management with package management tools are lost
Let's write <dependencies>
properly without throwing it to automatic generation.
If there are libraries with the same name but do not match (with differences) You should explicitly control which one is loaded:
--Upload to private repository as another version --Or, control on Gradle side (below)
repositories {
maven { // Maven central
url 'http://central.maven.org/maven2/'
content {
excludeGroup 'com.sun.jmx' //Don't get this group from here
excludeModule 'jdom', 'jdom' //Don't get this library from here
}
//MavenCentral to make the above description()I do not use the abbreviated notation such as
}
Next Step
What you especially want to do when you include a package management tool:
--Version Collision detection --Duplicate detection on classpath --Vulnerability information check by OWASP dependency check
A deeper explanation on the JJUG 2018 Spring slides:
--Title: How to make a compromise between Spring Boot and general libraries
-Speaker Deck and Qiita The same content is posted on
This time, the process when the package management tool was installed:
--Define goals ――Draw a vision of how package management will improve --Build a strategy ――What to do for realization ・ What do you dare not to do? --Solving problems with techniques: Various ideas mentioned this time --Utilization of Gradle --Jar diff check --Utilization of private repository
Even in situations where you may think that the darkness is so deep that you can't touch it __ Set the appropriate scope and realize with technology __ to move forward.
Recommended Posts