Introduction

Continuing from the previous tutorial, when I searched for materials from the desire to touch SageMaker, I created an image judgment Web application using machine learning Hands-on materials /article/column/ai-ml01.html) was found, so I decided to implement it. However, I gave up because an error occurred during the execution of SageMaker and I could not solve it. This article describes the error event that occurred.

Below is an overview of the architecture that was planned to be built.

(Quoted from http://www.intellilink.co.jp/article/column/ai-ml01.html)

In addition, the following is a completed image of the Web application to be developed. It is a mechanism to select the image you want to judge on the left side of the screen and display the judgment result of what is reflected in the image on the right side of the screen.

(Quoted from http://www.intellilink.co.jp/article/column/ai-ml01.html)

Main story

Refer to Hands-on materials --Create S3 --Implementation of machine learning part by SageMaker

However, when I ran SageMaker, I confirmed that an error occurred.

When I check the log, an error is displayed in the following execution part.

%%bash

mkdir -p caltech_256_train_60
for i in 256_ObjectCategories/*; do
    c=`basename $i`
    mkdir -p caltech_256_train_60/$c
    for j in `ls $i/*.jpg | shuf | head -n 60`; do
        mv $j caltech_256_train_60/$c/
    done
done

python im2rec.py --list --recursive caltech-256-60-train caltech_256_train_60/
python im2rec.py --list --recursive caltech-256-60-val 256_ObjectCategories/

The error wording is as follows.

ls: cannot access 256_ObjectCategories/*/*.jpg: No such file or directory

As I continued to check, the previous execution part output the following message, although it was not interrupted. The following is the execution part.

import os
import urllib.request

def download(url):
    filename = url.split("/")[-1]
    if not os.path.exists(filename):
        urllib.request.urlretrieve(url, filename)


# Caltech-256 image files
download('http://www.vision.caltech.edu/Image_Datasets/Caltech256/256_ObjectCategories.tar')
!tar -xf 256_ObjectCategories.tar

# Tool for creating lst file
download('https://raw.githubusercontent.com/apache/incubator-mxnet/master/tools/im2rec.py')

The message is below.

tar: This does not look like a tar archive
tar: Skipping to next header
tar: Exiting with failure status due to previous errors

The tar decompression command is being executed, but the processing is skipped because the target file is not in tar format.

--Isn't it a tar file? → I checked the target, but it was a tar file --Isn't it possible to download due to authority etc.? → I ran it without the tar command, but I was able to download it without any problems. --Is the tar command option wrong? The options for this tar command are as follows. x: Unzip the archived file and extract / restore it f: Specify the archive file name → When I removed the command f and executed it, the above message was no longer output, but it never ended.

When I looked it up, it seems that it may not be able to decompress properly if it is tar.gz, but this time gzip compression is not done. .. .. Give up and suspend.

[LINUX] tar command error (unresolved)

Introduction

Main story