How to handle uploaded images

Introduction

Where is the right place to store images uploaded by users when creating a web service? ?? It's a service that requires uploading like Instagram or pixiv, and even if it doesn't, I think that you often upload photos for your profile.

Tobidemo When I made it, I was having trouble storing the images uploaded by the user. Once you know the path where the image is saved, you can follow the path and see other images as much as you want! ?? Somehow it feels bad. I tried to find out, but there was no article like best practice, so I have summarized the method I investigated and implemented in my own way.

Investigation

I checked some of the ways that web services in the world process images.

pixiv It seems that illustrations and icons are managed by date and time information + UUID. If the number of directories increases every day, it will become 1825 directories in 5 years, but the search performance around here is I am wondering how to improve it.

An illustration:

https: // i.pximg.net/img-original/img/ ** Date and time information ** / 05/19 / ** Date and time information ** / 32/55/608090000_p0.jpg

icon:

https: //i.pximg.net/user-profile/img/** Date and time information ** / 02/20 / ** Date and time information ** / 52/02/8996319_4b86000332fa62d00000392d64a00000_170.jpg;

instagram Instagram seems to manage a combination of UUIDs. The path is long.

icon:

https: //scontent-nrt1-1.cdninstagram.com/vp/26223423_148400000808676_7148163548756000000/5BC4422C/t51.2885-19/s150x150/26223423_148400000808676_7148163548756000000_n.jpg

Photo:

https:// scontent-nrt1-1.cdninstagram.com/vp/26223423_148400000808676_7148163548756000000/t51.2885-15/s640x640/sh0.08/e35/31232925_800900000769909_8151073854442100000_n.jpg

Zoho Docs (cloud storage)

The above two services can download images by entering the URL in the browser, but It couldn't be displayed in Zoho Docs. The icon can be displayed. Of course, it goes without saying that private images are authenticated when they are displayed.

Photo: https:// download.zoho.com/docs/prv/00000c661d00000f644738ec6a58018300000?_t=1526805269750

icon: https:// accounts.zoho.com/file?fs=normal&nocache=10000805400000

Survey results

Images uploaded by users are saved using UUID, date, etc. as paths. For services that are supposed to be shared, such as image sharing services, It is difficult to predict because UUID etc. are used for the path. (Since it is public information in the first place, there is no problem even if it is predicted)

Services that are supposed to be managed privately perform some kind of authentication for image display.

It was a very natural result, but I found out how to store images in general.

Implementation in Tobidemo

Based on this information, consider the image upload function & display part. Tobi Demo is a tool for creating software demos. In some cases, images that can be published to a third party may be uploaded, and in BtoB, it is expected that only specific people will want to share them. Currently, anyone who knows the URL can access the demo, but in the future we will limit access to each demo. I was also expecting to put it on. Therefore, I assumed that it contains images that cannot be released to the public.

Image storage

The image is saved in a location that is not accessible from outside the server local with the following path. If you have not registered, the image will be saved in tmp and will be deleted in 30 days. (Delete function is not implemented) If you have registered as a user, the directories are separated using the user ID and demo ID. The image was imitated to generate and use a 6-digit UUID.

tmp/UUID.png User ID / Demo ID / UUID.png

It was uncomfortable to store public NG images in a directory that can be published externally, so In the first place, I decided to use a storage method that is inaccessible even if I know the path.

Image display

スクリーンショット 2018-05-20 18.36.46.png

Instead of returning the URL when there is an image request By converting the image to a Byte column and having the browser read it as it is, Image data can now be displayed without passing a URL.

You don't have to think about accessing the image with this, so if necessary, It seems that the access control function can be implemented by setting the authentication to be performed when the demo link is called.

Well, honestly I'm wondering if this was correct. It might have been good to put only the images (demo) that need to be authenticated in a separate directory and authenticate there. I wonder if there is no problem with the load when passing the Byte column directly to the browser, I feel that it was not the best. (Even if you pass the link, the data will be downloaded and read anyway, so it shouldn't change?) Is there a book that summarizes the knowledge about image processing of web services? ..

The following is an implementation that converts to a Byte column and passes it when displaying an image.

AddImgToDropZoneList.java




					fileStream = new FileInputStream(user_folder + "/" + dropzoneList.get(i).getImg() + ".png ");

					ByteArrayOutputStream os = new ByteArrayOutputStream();
					byte[] indata = new byte[10240 * 16];
					int siz;

					try {
						while ((siz = fileStream.read(indata, 0, indata.length)) > 0) {
							os.write(indata, 0, siz);
						}
					} catch (Exception e) {
						//TODO auto-generated catch block
						e.printStackTrace();
						System.out.print("failed to read byte stream");
						fileStream.close();

					}

					String base64 = new String(Base64.encodeBase64(os.toByteArray()), "ASCII");

					// System.out.print(base64);

					data = new StringBuffer();
					data.append("data:image/png;base64,");
					data.append(base64);
					fileStream.close();

					dropzoneWithImglist.add(new DropZoneWithImgBean(dropzoneList.get(i).getId(),
							dropzoneList.get(i).getLinkid(), dropzoneList.get(i).getDropzone(),
							dropzoneList.get(i).getImg(), data.toString(), width, height));
				} else {
					dropzoneWithImglist.add(new DropZoneWithImgBean(dropzoneList.get(i).getId(),
							dropzoneList.get(i).getLinkid(), dropzoneList.get(i).getDropzone(),
							dropzoneList.get(i).getImg(), "./../../img/default-none.png ", 1300, 1000));
				}

Summary

I thought that the images were stored locally, so when I chose VPS, I chose a server with plenty of HDD. Everyone's images are stored in S3. I'm glad I had to store it locally this time, but now let's store the image in S3.

Recommended Posts

How to handle uploaded images
[rails] How to post images
How to minimize Java images
How to handle an instance
Handle CSV files uploaded to GCS
[Rails] How to delete images uploaded by carrierwave (using devise)
[Rails] How to upload images using Carrierwave
[java] Summary of how to handle char
How to switch thumbnail images with JavaScript
How to deploy
How to handle sign-in errors with devise
[Rails] How to handle data using enum
[java] Summary of how to handle character strings
How to save images on Heroku to S3 on AWS
[Rails] How to upload multiple images using Carrierwave
How to handle optional in Protocol Buffers (proto3)
How to link images using FactoryBot Active Storage
How to post images on Heroku + CarrierWave + S3
How to develop OpenSPIFe
How to use Map
How to write Rails
How to use rbenv
How to use letter_opener_web
How to use with_option
How to use fields_for
How to use java.util.logging
How to use map
How to use collection_select
How to adapt Bootstrap
How to use Twitter4J
How to use active_hash! !!
How to install Docker
How to use MapStruct
How to use hidden_field_tag
How to use TreeSet
How to write dockerfile
How to uninstall Rails
How to install docker-machine
[How to use label]
How to make shaded-jar
How to write docker-compose
How to use identity
How to use hashes
How to write Mockito
How to create docker-compose
How to use JUnit 5
How to install MySQL
How to write migrationfile
How to build android-midi-lib
How to use Dozer.mapper
How to use Gradle
How to use org.immutables
How to use java.util.stream.Collector
How to use VisualVM
How to use Map
How to install ngrok
How to type backslash \
How to concatenate strings
How to delete untagged images in bulk with Docker
How to handle TSV files and CSV files in Ruby
How to download images from AWS S3 (rails, carrierwave)