I made a tool to output the difference of CSV file

Although there are many tools out there that compare file differences I didn't have what I wanted I made a tool to output the difference between two CSV files.

Specify the key line number in the argument, It is a simple tool that outputs to another line if the key value is different.

CSVDiff.java



import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.nio.charset.Charset;
import java.sql.Timestamp;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

/**
 *CSV differential output
 */
public class CSVDiff {

    private String split = null;

    /**
     *Delimiter
     */
    private String delimiter = null ;

    /**
     *Key item index
     ** 0 Start
     */
    private List<Integer> keyIndex = null;

    /**
     *Import file placement folder
     */
    private String inputDir = null ;

    /**
     *Import file 1
     */
    private String inputFile1 = null ;

    /**
     *Import file 2
     */
    private String inputFile2 = null ;

    /**
     *Difference file output folder
     */
    private String outputDir = null ;

    /**
     * */
    private List<String[]> recordList1 = null;

    /**
     * */
    private List<String[]> recordList2 = null;

    private List <String>  outputList  = null ;

    /**
     *constructor
     * @param delimiterType Delimiter type (1: Other than comma 1: Tab)
     * @param args Key item index
     * @param inputDir Import file placement folder
     * @param inputFile1 Import file 1
     * @param inputFile2 Import file 2
     * @param outputDir Difference file output folder
     */
    public CSVDiff(String delimiterType, String args, String inputDir, String inputFile1, String inputFile2,String outputDir) {

        //================================================================================
        //1. 1. Delimiter type judgment
        //================================================================================
        if ("1".equals(delimiterType)) {
            this.delimiter = ",";
        } else {
            this.delimiter = "\t";
        }
        this.split = "*" + this.delimiter + "*" + this.delimiter + "*";

        //================================================================================
        //2. Convert key item index when comparing records to array
        //================================================================================
        this.keyIndex = new ArrayList<Integer> ();
        String arr[] = args.split(",");
        for (String item : arr) {
            int index = Integer.parseInt(item);
            if (! this.keyIndex.contains(index) ) {
                this.keyIndex.add(index);
            }
        }

        //================================================================================
        //3. 3. Set various folder names and file names
        //================================================================================
        this.inputDir   = inputDir;
        this.outputDir  = outputDir;
        this.inputFile1 = inputFile1;
        this.inputFile2 = inputFile2;
    }

    /**
     *
     * @param outputType
     * @param array1
     * @param array2
     * @return
     */
    private String makeRecord (int outputType,String[] array1 ,String[] array2) {
        StringBuilder outputRecord = new StringBuilder() ;

        if (outputType == 0) {
            //================================================================================
            //Record 1 = Record 2
            //→ Output record 1 and record 2
            //================================================================================

            //--------------------------------------------------------------------------------
            //Record 1 output
            //--------------------------------------------------------------------------------
            for ( int i=0;i<array1.length;i++) {
                if (i == 0) {
                    outputRecord.append(array1[i]);
                } else {
                    outputRecord.append(this.delimiter + array1[i]);
                }
            }
            //--------------------------------------------------------------------------------
            //Delimiter output
            //--------------------------------------------------------------------------------
            outputRecord.append(this.delimiter);
            outputRecord.append(split);

            //--------------------------------------------------------------------------------
            //Record 2 output
            //--------------------------------------------------------------------------------
            for ( int i=0;i<array2.length;i++) {
                outputRecord.append(this.delimiter + array2[i]);
            }

        } else if (outputType == 1) {
            //================================================================================
            //Record 1 <Record 2
            //→ Output only record 1
            //================================================================================

            //--------------------------------------------------------------------------------
            //Record 1 output
            //--------------------------------------------------------------------------------
            for ( int i=0;i<array1.length;i++) {
                if (i == 0) {
                    outputRecord.append(array1[i]);
                } else {
                    outputRecord.append(this.delimiter + array1[i]);
                }
            }
            //--------------------------------------------------------------------------------
            //Delimiter output
            //--------------------------------------------------------------------------------
            outputRecord.append(this.delimiter);
            outputRecord.append(split);

            //--------------------------------------------------------------------------------
            //Record 2 output (comma or tab only)
            //--------------------------------------------------------------------------------
            for ( int i=0;i<array1.length;i++) {
                outputRecord.append(this.delimiter);
            }

        }  else if (outputType == 2) {
            //================================================================================
            //Record 1> Record 2
            //→ Output only record 2
            //================================================================================

            //--------------------------------------------------------------------------------
            //Record 1 output (comma or tab only)
            //--------------------------------------------------------------------------------
            for ( int i=0;i<array2.length;i++) {
                outputRecord.append(this.delimiter);
            }
            //--------------------------------------------------------------------------------
            //Delimiter output
            //--------------------------------------------------------------------------------
//            outputRecord.append(this.delimiter);
            outputRecord.append(split);

            //--------------------------------------------------------------------------------
            //Record 2 output
            //--------------------------------------------------------------------------------
            for ( int i=0;i<array2.length;i++) {
                outputRecord.append(this.delimiter + array2[i]);
            }
       }

        return outputRecord.toString();

    }


    /**
     *CSV difference list output
     * @return 0: Normal termination 1: Abnormal termination
     */
    protected int outputCSVDiff ()  {
        boolean isError = false ;

        // --------------------------------------------------------------------------------
        //(1) Import / output file, folder existence check
        // --------------------------------------------------------------------------------
        if (this.checkFileExists() != 0) {
            System.err.println("File or folder for import / output does not exist");
            return 1;
        }

        // --------------------------------------------------------------------------------
        //(1) Read file
        // --------------------------------------------------------------------------------
        if (this.readFile() != 0) {
            System.err.println("File read error");
            return 1;
        }

        // --------------------------------------------------------------------------------
        //(2) Output list generation
        // --------------------------------------------------------------------------------
        this.outputList = new ArrayList<String> () ;
        int i_file1 = 0 ;
        int i_file2 = 0 ;

        while (i_file1 < this.recordList1.size() && i_file2 < this.recordList2.size()) {
            String[] record1 = this.recordList1.get(i_file1);
            String[] record2 = this.recordList2.get(i_file2);
            String   outputRecord = null ;


            if (record1.length != record2.length) {
                System.err.println("There is a difference in the number of records between file 1 and file 2.");
                isError = true ;
                break ;
            }

            //========================================================================
            //Key item match judgment
            //========================================================================
            int diffType = 0 ;
            for (int keyIndex :this.keyIndex) {

                if (record1[keyIndex].compareTo(record2[keyIndex]) < 0 ) {
                    //File 1 record <File 2 record
                    diffType = 1;
                    break;
                } else if (record1[keyIndex].compareTo(record2[keyIndex]) > 0 ) {
                    //File 1 record> File 2 record
                    diffType = 2;
                    break;
                }
            }

            if (diffType == 0) {
                //File 1 record = File 2 record
                outputRecord = this.makeRecord(diffType, record1, record2) ;
                System.out.println(outputRecord);
                this.outputList.add(outputRecord);
                i_file1 ++;
                i_file2 ++;

            } else if (diffType == 1) {
                //File 1 record <File 2 record
                outputRecord = this.makeRecord(diffType, record1, null);
                System.out.println(outputRecord);
                this.outputList.add(outputRecord);
                i_file1 ++;
            } else {
                //File 1 record> File 2 record
                outputRecord = this.makeRecord(diffType, null, record2);
                System.out.println(outputRecord);
                this.outputList.add(outputRecord);
                i_file2 ++;
            }
        } // end-of-while

        if (isError) { return 1; }

        while (i_file1 < this.recordList1.size() ) {
            String[] record1 = this.recordList1.get(i_file1);
            String outputRecord = this.makeRecord(1, record1, null);
            System.out.println(outputRecord);
            this.outputList.add(outputRecord);
            i_file1 ++;
        }

        while (i_file2 < this.recordList2.size()) {
            String[] record2 = this.recordList2.get(i_file2);
            String outputRecord = this.makeRecord(2, null, record2) ;
            System.out.println(outputRecord);
            this.outputList.add(outputRecord);
            i_file2 ++;
        }

        // --------------------------------------------------------------------------------
        //(3) File output
        // --------------------------------------------------------------------------------
        this.writeFile();


         return 0 ;
    }

    private void writeFile () {
        FileOutputStream fos  = null;
        OutputStreamWriter osw = null;
        // --------------------------------------------------------------------------------
        //(1) Output file time stamp generation
        // --------------------------------------------------------------------------------
        Timestamp timestamp    = new Timestamp(System.currentTimeMillis());
        SimpleDateFormat sdf   = new SimpleDateFormat("yyyyMMddHHmmss");
        String fileName = "csvdiff_"+sdf.format(timestamp)+".txt" ;

        try {
            fos = new FileOutputStream(this.outputDir+"/"+fileName);
            osw = new OutputStreamWriter(fos,"Shift_JIS");

            for (String record : this.outputList) {
                osw.write(record + "\r\n") ;
            }
            osw.close();
            fos.close();
        } catch (IOException ex) {
            ex.printStackTrace();
        } finally {
            if (osw != null ) {
                try {
                    osw.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
            if (fos != null ) {
                try {
                    fos.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }


        }


    }

    /**
     *Check for the existence of import / output files and folders
     * @return 0: Import / output file, with folder 1: Import / output file, folder does not exist
     */
    private int checkFileExists () {

        // --------------------------------------------------------------------------------
        //(1) Import file 1 Existence check
        // --------------------------------------------------------------------------------
        File file1 = new File(this.inputDir+"/"+this.inputFile1) ;
        if (!file1.exists()) {
            System.err.println("error->Import file 1 ""+this.inputDir+"/"+this.inputFile1+"』Does not exist! !!");
            return 1;
        }

        // --------------------------------------------------------------------------------
        //(2) Import file 2 existence check
        // --------------------------------------------------------------------------------
        File file2 = new File(this.inputDir+"/"+this.inputFile2) ;
        if (!file2.exists()) {
            System.err.println("error->Import file 2 ""+this.inputDir+"/"+this.inputFile2+"』Does not exist! !!");
            return 1;
        }

        // --------------------------------------------------------------------------------
        //(3) Check the output file placement folder
        // --------------------------------------------------------------------------------
        File outputDir = new File(this.outputDir) ;
        if (!outputDir.exists()) {
            System.err.println("error->Output folder ""+this.outputDir+"』Does not exist! !!");
            return 1;
        }
        return 0 ;
    }


    /**
     *Read the file and set it in the array list
     * @return 0: Read OK 1: Read NG
     */
    private int readFile() {
        int retVal = 0 ;
        File file1 = new File(this.inputDir+"/"+this.inputFile1) ;
        File file2 = new File(this.inputDir+"/"+this.inputFile2) ;

        try {
            this.recordList1 = this.makeRecordList(file1);
            this.recordList2 = this.makeRecordList(file2);
        } catch (Exception e) {
            e.printStackTrace();
            retVal = 1;
        }
        return retVal;
    }

    /**
     *Refill the CSV file contents into an array list
     * @param file
     * @return
     * @throws Exception
     */
    private List<String[]> makeRecordList (File file) throws Exception {
        boolean isError = false;
        List<String[]> recordList  = new ArrayList<String[]>() ;

        FileInputStream   fis  = null;
        InputStreamReader isr  = null ;
        BufferedReader    br   = null ;

        try {
            fis   = new FileInputStream(file);
            isr   = new InputStreamReader(fis, Charset.forName("MS932"));
            br    = new BufferedReader(isr);

            int i = 1 ;
            String record = null ;
            while((record = br.readLine()) != null) {
                String[] recordArray = record.split(this.delimiter);

                System.out.println("["+i+"]Line:"+Arrays.toString(recordArray));
                recordList.add(recordArray);
//                System.out.println("["+i_file1+"]Line:"+record);
                i ++ ;
            }

        } catch(FileNotFoundException e ) {
            e.printStackTrace();
            isError = true;
        } catch(IOException e ) {
            e.printStackTrace();
            isError = true;
        } finally {
            if (br != null) {
                try {
                    br.close();
                } catch (IOException e) {
                    e.printStackTrace();
                    isError = true;
                }
            }
            if (isr != null) {
                try {
                    isr.close();
                } catch (IOException e) {
                    e.printStackTrace();
                    isError = true;
                }
            }
            if (fis != null) {
                try {
                    fis.close();
                } catch (IOException e) {
                    e.printStackTrace();
                    isError = true;
                }
            }
        }
        if (isError) { throw new Exception(); }

        return recordList ;

    }

The following is the execution class

CSVDiffExecute.java



/**
 *CSV differential output execution class
 *
 */
public class CSVDiffExecute {
    /**
     *Main method
     * @param args [0]Delimiter type (1: Other than comma 1: Tab)
     *              [1]Index of key items (multiple specifications can be specified separated by single-byte commas)
     *              [2]Import file placement folder
     *              [3]Import file 1
     *              [4]Import file 2
     *              [5]Difference file output folder
     */
    public static void main(String[] args) {
        //================================================================================
        //Argument check
        //================================================================================
        if (args.length != 6) {
            System.err.println("error->There are not enough arguments.");
            System.exit(1);
        }
        //================================================================================
        //CSV differential output execution
        //================================================================================
        CSVDiff csvDiff = new CSVDiff(args[0], args[1], args[2], args[3], args[4], args[5]);
        csvDiff.outputCSVDiff();
    }
}


Recommended Posts

I made a tool to output the difference of CSV file
I made a gem to post the text of org-mode to qiita
I want to output the day of the week
I learned about the existence of a gemspec file
I tried JAX-RS and made a note of the procedure
I made a package.xml generation tool.
When I switched to IntelliJ, I got a lot of differences in the encoding of the properties file.
I want to get a list of the contents of a zip file and its uncompressed size
[chown] How to change the owner of a file or directory
I made a GitHub Action that makes it easy to understand the execution result of RSpec
I made a new Java deployment tool
Find the difference from a multiple of 10
The story of forgetting to close a file in Java and failing
I made an app to scribble with PencilKit on a PDF file
Create a Tokyo subway map from the CSV file of station data.jp
How to find out the Java version of a compiled class file
I made a Diff tool for Java files
Make a margin to the left of the TextField
I tried to develop a man-hour management tool
Set the time of LocalDateTime to a specific time
I want to var_dump the contents of the intent
Output of how to use the slice method
I made a Docker container to run Maven
[Spring Batch] Output table data to CSV file
I was a little addicted to the S3 Checksum comparison, so I made a note.
I made a sample of how to write delegate in SwiftUI 2.0 using MapKit
I managed to get a blank when I brought the contents of Beans to the textarea
I want to recursively get the superclass and interface of a certain class
I want to get a list of only unique character strings by excluding fixed character strings from the file name
[Ruby] I want to make a program that displays today's day of the week!
I want to call a method of another class
[Java] I want to calculate the difference from the date
I tried to read and output CSV with Outsystems
I was addicted to the record of the associated model
I tried to summarize the state transition of docker
I tried to create a log reproduction script at the time of apt install
I tried to decorate the simple calendar a little
Convert the array of errors.full_messages to characters and output
I tried to reduce the capacity of Spring Boot
I want to display the name of the poster of the comment
I want to monitor a specific file with WatchService
JavaFX-Specify the location of the CSS file relative to the fxml file
The CSV file that I was able to download suddenly started to appear on the page.
I wrote a sequence diagram of the j.u.c.Flow sample
[Rails] I want to display the link destination of link_to in a separate tab
I want to find the MD5 checksum of a file in Java and get the result as a string in hexadecimal notation.
A memorandum because I was addicted to the setting of the Android project of IntelliJ IDEA
I want to be aware of the contents of variables!
I want to return the scroll position of UITableView!
I want to simplify the log output on Android
I tried to make a product price comparison tool of Amazon around the world with Java, Amazon Product Advertising API, Currency API (2017/01/29)
I want to add a delete function to the comment function
I made Togmarks, which allows the second year of junior high school to vote for their ideas in a week.
[VBA] I tried to make a tool to convert the primitive type of Entity class generated by Hibernate Tools to the corresponding reference type.
I want to find out which version of java the jar file I have is available
I tried to make the sample application into a microservice according to the idea of the book "Microservice Architecture".
How to request a CSV file as JSON with jMeter
I made the server side of an online card game ①
I made a server side of an online card game ⑤
I made a server side of an online card game ③
I tried to summarize the basics of kotlin and java