Try setting up Talend Open Studio on Amazon EC2 (ubuntu)

Overview

I needed to create a set of migration jobs for a certain service at the fastest speed, so I used talend on my local mac to create a migration job.

It's really easy to make and it's great! It was good up to that point, but all the materials were on the local PC, and in the end, maintenance became a troublesome thing unless we shared them.

I would like to build it on Amazon EC2 and move the job so that it can be taken over properly.

** Note: This article worked! It is not an article. ** **

** By adopting Linux Desktop, there was a lot of work to be done! It is an article with the content. I think we can meet the needs of just executing a job. ** **

Installation requirements

As far as I researched, it looks like this.

Does talend not work in amazon workspace? → talend forum

I noticed it later, but if you read it carefully, it seems to be a story limited to "for Big Data".

JDK1.7.x (** Note: jdk1.8.x is standard from talend 6.4 series **)
Ubuntu 14.4 (official Talend Data Presentetion AMI was this version)
Install desktop
talend 6.3 (version as of 2017/03. Job was created with this version ** Note: As of 2017/07/10, talend 6.4 series can be downloaded **)
t2.small or t2.medium

FYI

Talend Open Studio for ESB Installation Guide

The actual information is "for Data Integration", but it is for reference only.

procedure

Advance preparation

EC2 instance creation is omitted from the description. I used "ubuntu / images / hvm-ssd / ubuntu-trusty-14.04-amd64-server-20170619 (ami-8c4055eb)".

First connect. .. ..

chmod 400 ST_Talend01.pem
ssh -i "ST_Talend01.pem" [email protected]

Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 3.13.0-121-generic x86_64)

… (Omitted)

Package update

sudo apt-get update
sudo apt-get -y upgrade

Change to Japanese environment

Common character codes, locales, timezone settings, etc. Execute the command with root privileges.

sudo su
apt-get install -y language-pack-ja
update-locale LANG=ja_JP.UTF-8

timedatectl set-timezone Asia/Tokyo

To check, exit ssh and reconnect.

If it looks like the following, it's OK.

$ locale
LANG=ja_JP.UTF-8
LANGUAGE=
LC_CTYPE="ja_JP.UTF-8"
LC_NUMERIC="ja_JP.UTF-8"
LC_TIME="ja_JP.UTF-8"
LC_COLLATE="ja_JP.UTF-8"
LC_MONETARY="ja_JP.UTF-8"
LC_MESSAGES="ja_JP.UTF-8"
LC_PAPER="ja_JP.UTF-8"
LC_NAME="ja_JP.UTF-8"
LC_ADDRESS="ja_JP.UTF-8"
LC_TELEPHONE="ja_JP.UTF-8"
LC_MEASUREMENT="ja_JP.UTF-8"
LC_IDENTIFICATION="ja_JP.UTF-8"
LC_ALL=

$  timedatectl 
      Local time:Tue 2017-07-11 16:40:01 JST
  Universal time:Tue 2017-07-11 07:40:01 UTC
        Timezone: Asia/Tokyo (JST, +0900)
     NTP enabled: yes
NTP synchronized: no
 RTC in local TZ: no
      DST active: n/a

** Addition **

http://uyama.coffee/wp/【決定版】aws-ec2でリモートデスクトップ/ According to this, the following packages were additionally installed.

$ sudo apt-get install ibus-anthy
$ sudo apt-get install scim-anthy
$ sudo apt-get install fcitx-anthy

1. Install ubuntu desktop

Remote desktop connection to amazon ec2 ubuntu machine from windows

With reference to this, I carried out the procedure.

2. Allow remote connection to ubuntu desktop

Prerequisites

Inbound of the following ports is allowed in the security group settings.

type	port
RDP	3389
VNC	5901

** When setting up a security group, be sure to specify the IPs that allow inbound. ** **

mac

Execute the vncserver command on the server side.
Set the connection password.

# vncserver

You will require a password to access your desktops.

Password:
Verify:
Password too long - only the first 8 characters will be used
xauth:  file /root/.Xauthority does not exist

New 'ip-XXX-XXX-XXX-XXX:1 (root)' desktop is ip-XXX-XXX-XXX-XXX:1

Creating default startup script /root/.vnc/xstartup
Starting applications specified in /root/.vnc/xstartup
Log file is /root/.vnc/ip-XXX-XXX-XXX-XXX:1.log

Change the vncserver settings.

rm ~/.vnc/xstartup
ln -s /etc/X11/Xsession ~/.vnc/xstartup

Connect with ** hostname: 5901 ** using screen sharing .app.

It is like this.

FYI

I referred to the following page in part.

A story about automating browser operations by putting a GUI in Ubuntu on AWS EC2

windows

apt-get install xrdp
 * Starting Remote Desktop Protocol server 

apt-get install xfce4

sudo vim /etc/xrdp/xrdp.ini

[xrdp1]
name=sesman-Xvnc
lib=libvnc.so
username=ask
password=ask
ip=127.0.0.1
#port=-1
port=ask-1

OK if you can check the connection with the remote desktop of windows

Remote connection is not possible after restarting. Both RDP and vnc.

https://futuremix.org/2009/01/linux-vnc-server-install Set vncserver to start automatically

trouble shooting

** Q. I was able to connect, but nothing is displayed ... **

After restarting Xubuntu 14.04, I couldn't see the icon on my desktop, I couldn't right-click, and I couldn't change the wallpaper from the default one (it doesn't take effect).

At first, I wondered if I had set the desktop to not display icons in Settings> Desktop> Menu, but that doesn't seem to be the case. Rebooting doesn't fix it, and it's likely that the service that provides desktop functionality hasn't started.

The solution is in terminal

$ xfdesktop

Will bring the desktop back to life.

3. Install java

sudo add-apt-repository ppa:webupd8team/java
sudo apt update
sudo apt install oracle-java8-installer

Failure example

** I put jdk7, but java8 became standard from talend 6.4 series **

sudo apt-get install openjdk-7-jdk

Verification.

# java -version
java version "1.7.0_131"
OpenJDK Runtime Environment (IcedTea 2.6.9) (7u131-2.6.9-0ubuntu0.14.04.2)
OpenJDK 64-Bit Server VM (build 24.131-b00, mixed mode)

4. Install talend

Connect to the linux desktop remotely and download "Talend Open Studio for BigData 20170623_1246-V6.4.1" from the talend site.

「Open Studio for Data Integration」

Create "/ usr / local / talend" folder and unzip to the same folder.
Run TOS_DI-linux-gtk-x86_64.

Up to this point, talend has been started.

I regret that the stuffing was sweet

I chose ubuntu + xfce4 without any particular concern, but I ran too far. .. .. There are various inconveniences, so what should I do?

Insufficient auto-start settings

vncserver
xfdesktop

Japanese input does not work on mac Japanese keyboard

… Even if the mac alone feels good, can a successor be used…

I want to change the screen resolution to "1440 x 900", but I can't

I ran into a problem peculiar to xfce, and the result seems to be better to review the desktop settings.

It's hard to understand at first glance, but it's quite fatal that the screen area is too small for creating a job with talend.

ubuntu + xfce

Specify the monitor resolution in the initial process

It may be OK if I deal with this area, but I was planning to complete it in one day, so the time has expired.

Another voice! If you crush it, it will be useful, but in my case, I realized that it was out of the purpose of "can be used by other people".

I think it will be rebuilt into a windows-based amazon workspace.