Monolithic application refactoring

The application developed in-house is ["Big ball of mud"](https://ja.wikipedia.org/wiki/%E5%A4%A7%E3%81%8D%E3%81%AA%E6%B3 % A5% E3% 81% A0% E3% 82% 93% E3% 81% 94) I faced a problem that development efficiency did not improve because it was in the state. So I decided to refactor. I am refactoring while studying books such as "Clean Architecture", "Microservice Architecture", and "Refactoring". It's not finished yet, but I'll summarize what I've been thinking about when refactoring.

Organize the current status of monolithic applications

I tried to organize the state of the application before refactoring. I have the following problems.

--Multiple subsystems and components are managed collectively in one git repository (source code amount is about 700,000 lines) ――This is the main problem. With this configuration, even if the function development is closed to some subsystems, you will end up building and testing all the modules with CI. --One DB is shared by all subsystems --Slight changes to the DB table can affect all subsystems. ――It takes time to investigate which table should be changed to achieve what you want to achieve in function development. --Automated testing is too tightly coupled with features other than what you want to verify --The test code depends on the details of the lower module. Since the test is implemented in the position like the integration test, various tests can be done by modifying the product code of the lower module. --There is also a test that loads DB data, and maintenance costs are high. ――It takes about 2 hours to build and test with one CI --The main reason is that there is only one git repository and the unit test code is implemented in a position like an integration test. The disadvantages of hindering development agility outweigh the advantages of quality assurance through automated testing. --The lower module contains too much detail --Because the lower module contains details, the lower module will be updated frequently in function development. In addition, the upper module depends on the details because the details exist in the lower module. As a result, it cannot be closed to the fix.

Refactoring to the problem

--Batch management of multiple subsystems and components in one git repository --Currently, multiple maven projects divided by "functional unit" are stored in one git repository, but according to the single responsibility principle Separate and divide into another repository.

--One DB is shared by all subsystems --By sharing the DB, one table is packed with multiple interests. Therefore, the number of columns in the table is inevitably huge. This is divided into a table in which only the necessary columns are extracted for each subsystem, and the schema is divided for each subsystem.

--Automated testing is too tightly coupled with features other than what you want to verify --Understand the external specifications from the existing test code, and refactor the existing implementation while writing the test code in a form that depends on the abstraction of the lower module.

――It takes about 2 hours to build and test with one CI --Refactoring git repository centralized management, tight coupling between unit test code and implementation will solve the problem

--The lower module contains too much detail -[Dependency Inversion Principle](https://ja.wikipedia.org/wiki/%E4%BE%9D%E5%AD%98%E6%80%A7%E9%80%86%E8%BB% Refactored to apply A2% E3% 81% AE% E5% 8E% 9F% E5% 89% 87). The lower modules basically aim to consist only of abstract classes and interfaces.

Estimated effect of refactoring

--Batch management of multiple subsystems and components in one git repository --Assuming that the build test range for source code changes is about half, the build test time is simply halved. Assuming an existing build test time of 2 hours and an average CI of 5 times per day, 5 (builds) x 2 x 0.5 (time reduced by refactoring) x 20 (business days) = 100 hours / month cost reduction can be expected per developer.

--One DB is shared by all subsystems --Confirmation of no impact in function development that involves DB changes-Assuming that it takes 16 hours (2 man-days) to investigate DB changes, this will be 2 hours after DB partitioning. Assuming that function development with DB change occurs 3 times a month, 3 (number of function development occurrences) x 14 (time reduced by refactoring) = 42 hours / month cost reduction per developer You can expect it.

--Automated testing is too tightly coupled with features other than what you want to verify --Assuming that it takes 5 hours to modify the automated test, which is 1 hour after refactoring. Assuming that the opportunity to modify automated tests occurs 5 times a month, 5 (number of feature developments) x 4 (time reduced by refactoring) = 20 hours / month cost savings per developer You can expect it.

--The lower module contains too much detail --This affects the update frequency of lower modules. If the lower module is composed only of abstraction, it is possible to avoid the situation of building all the upper modules. Assume that the lower module is updated 10 times a month, and after refactoring it will be once a month. Assuming a source code build test time of 2 hours, 2 (build test time) x 9 (update frequency reduced by refactoring) = 18 hours / month cost reduction per developer Can be expected.

What can be considered from the trial calculation

None of the above estimates are accurate. Such a trial calculation cannot be made accurately in the first place. However, you must think about how to convey the benefits of doing this refactoring and get permission from your superiors to do the refactoring. And most importantly, ** the costs that can be reduced by the refactoring estimated above will continue to be paid forever unless refactored, and the costs will continue to increase. ** **

The cost that can be reduced by refactoring is estimated, but this is the reason why the cost for refactoring is not estimated. Costs that can be reduced by refactoring can only be reduced by refactoring. So if the cost of refactoring isn't realistic (such as 100 man-months), I think it should be done. Even though the cost will continue to increase, it seems nonsense to argue that if the cost of refactoring is 2 man-months, it will be done, and if it is 3 man-months, it will not be done.

What I felt after refactoring

――You can get the knowledge and experience you need as an architect at a high density. --"A good architect can delay the implementation decision as much as possible" Clean Architecture ? _encoding = UTF8 & btkr = 1) is mentioned. In order to achieve this, refactoring involves thinking and applying the necessary ** "dependency control", "module management", and "coupling" **. And you can experience the effect for yourself.

――You can grow into an engineer required in the field --Software continues to change. Because business demands continue to change. As a result, there are overwhelmingly more situations in which existing applications are extended than in situations where the power to create applications from scratch is demonstrated. Therefore, engineers who can improve development efficiency and reduce maintenance costs without destroying applications that are already running are indispensable in any market. The skills required of such engineers can be obtained through refactoring.

Reference materials for refactoring

-Clean Architecture --Microservices Architecture -[Database Refactoring](https://www.amazon.co.jp/%E3%83%87%E3%83%BC%E3%82%BF%E3%83%99%E3%83%BC%E3 % 82% B9% E3% 83% BB% E3% 83% AA% E3% 83% 95% E3% 82% A1% E3% 82% AF% E3% 82% BF% E3% 83% AA% E3% 83 % B3% E3% 82% B0-% E3% 82% B9% E3% 82% B3% E3% 83% 83% E3% 83% 88-W-% E3% 82% A2% E3% 83% B3% E3 % 83% 96% E3% 83% A9% E3% 83% BC / dp / 4894715007 / ref = cm_cr_arp_d_product_top? ie = UTF8) -[Refactoring](https://www.amazon.co.jp/%E3%83%AA%E3%83%95%E3%82%A1%E3%82%AF%E3%82%BF%E3% 83% AA% E3% 83% B3% E3% 82% B0% E2% 80% 95% E6% 97% A2% E5% AD% 98% E3% 81% AE% E3% 82% B3% E3% 83% BC% E3% 83% 89% E3% 82% 92% E5% AE% 89% E5% 85% A8% E3% 81% AB% E6% 94% B9% E5% 96% 84% E3% 81% 99% E3% 82% 8B% E2% 80% 95-OBJECT-TECHNOLOGY-Martin-Fowler / dp / 427405019X / ref = sr_1_1? s = books & ie = UTF8 & qid = 1553615256 & sr = 1-1 & keywords =% E3% 83% AA% E3% 83 % 95% E3% 82% A1% E3% 82% AF% E3% 82% BF% E3% 83% AA% E3% 83% B3% E3% 82% B0)