Handle CSV files uploaded to GCS

Read csv file

You can get the contents of the file with blob.getContent, but it will be a problem if the file size is large.

Bucket bucket = getBucket(bucketName);
Storage.BlobListOption option = Storage.BlobListOption.prefix(directory);

Page<Blob> blobs = bucket.list(option);
Iterator<Blob> iterator = blobs.iterateAll().iterator();

List<String> csvList = new ArrayList<>();
while (iterator.hasNext()) {
    Blob blob = iterator.next();
    byte[] content = blob.getContent(Blob.BlobSourceOption.generationMatch());
    csvList.add(new String(content));
}

for (String csv : csvList) {
    // process...
}

Read the csv file line by line

Instead of getContent, you can read and process line by line as follows

ReadChannel readChannel = blob.reader();
BufferedReader br = new BufferedReader(Channels.newReader(readChannel, "UTF-8"));

String line;
while((line = br.readLine()) != null){
    // process...
}

Recommended Posts

Handle CSV files uploaded to GCS
How to handle uploaded images
How to handle TSV files and CSV files in Ruby
Add files to jar files
Introduction to JAR files
Convert large XLSX files to CSV with Apache POI
Converting TSV files to CSV files (with BOM) in Ruby
Handle files with NIO.2.
How to handle an instance
How to rollback migration files