Access the in-memory data grid Apache Ignite from a Java client
Introduction
I started Apache Ignite to study in-memory data grids. First, install it, start it on 1 server / 2 servers, and try to use it as a cache server from a Java program.
I have also created a collection of links to the documentation on Apache Ignite below, so I hope you can refer to it.
-Links to documentation about Apache Ignite
What is Apache Ignite?
Apache Ignite is middleware that realizes an in-memory data grid. In-memory data grid is a technology that distributes a large amount of data in memory on multiple servers. A major feature is that it can be speeded up by handling data in-memory. One use is the distributed cache of a database that stores a large amount of data.
With Apache Ignite centered around the in-memory data grid, it can also play the following roles:
- In-memory database / distributed database
Treat the in-memory data grid as a database. Read / Write is decentralized, easy to scale out, and is faster than RDBs such as Oracle. The current version does not handle transactions, but it seems that future versions will support it. It also retains data in memory but can be persisted to the file system (HDFS). It is also stored in memory with a structure similar to HDFS, and is positioned as a subset of files on HDFS.
- KVS
A simple API for KVS is provided and can be used as a distributed KVS.
- Database caching
Databases such as Oracle and Cassandra / NoSQL can be used for caching.
- etc
For more information, see Apache Ignite Use Cases (https://ignite.apache.org/usecases.html).
In addition, we have created a collection of links for various information below for your reference.
Links to documentation about Apache Ignite
Execution environment
With Vagrant, I prepared a CentOS 7 environment and ran Apache Ignite. The procedure up to the construction of CentOS 7 is omitted because there are other sites that are easy to understand.
Try a single configuration and a cluster configuration (2 units). There are the following two servers.
・ 192.168.20.71 igniteserver1 ・ 192.168.20.72 igniteserver2
OpenJDK installation
Apache Ignite requires a JDK and supports Oracle JDK and Open JDK. This time we will use Open JDK.
# yum -y install java-1.8.0-openjdk-devel
# java -version
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
I have set JAVA_HOME and added the Java path to my PATH.
# echo "export JAVA_HOME=$(readlink -e $(which java)|sed 's:/bin/java::')" > /etc/profile.d/java.sh
# echo "PATH=\$PATH:\$JAVA_HOME/bin" >> /etc/profile.d/java.sh
# source /etc/profile
Install Apache Ignite
Download the module from the Apache Ignite site (https://ignite.apache.org/). Select Binary Releases below.
This time, I downloaded the latest version, v2.6.0 below.
Upload the "apache-ignite-fabric-2.6.0-bin.zip" downloaded earlier to the server, and extract it under "/ opt" with the following command.
# cd /opt
# unzip /tmp/apache-ignite-fabric-2.6.0-bin.zip
# ln -s /opt/apache-ignite-fabric-2.6.0-bin/ /opt/apache-ignite
To start Apache Ignite, execute the following command.
cd /opt/apache-ignite/bin
./ignite.sh
With no arguments, the default configuration file in "/opt/apache-ignite/config/default-config.xml" is read and started.
If the startup is successful, the following will be output to the console. When servers = 1, you can confirm that one server is running.
[22:42:07] Ignite node started OK (id=9ae050e2)
[22:42:07] Topology snapshot [ver=1, servers=1, clients=0, CPUs=1, offheap=0.74GB, heap=1.0GB]
The default configuration file is as follows. I will write the necessary settings under the "\ <bean id =" grid.cfg "" element, but the default settings are empty.
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<!--
Alter configuration below as needed.
-->
<bean id="grid.cfg" class="org.apache.ignite.configuration.IgniteConfiguration"/>
</beans>
Access Apache Ignite cache from Java (1 unit configuration)
Let's use Apache Ignite started earlier as a cache server from a Java application. Java application utilizes Maven project and adds the following dependency to pom.xml.
<dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-core</artifactId>
<version>2.6.0</version>
</dependency>
The Java client was created as follows by referring to the Apache Ignite sample program. It is a program that puts data to the cache called "" put-get-example "" on Apache Ignite, and gets and displays the same data as it is.
SimpleCacheApp.java
import org.apache.ignite.Ignition;
import org.apache.ignite.client.ClientCache;
import org.apache.ignite.client.ClientException;
import org.apache.ignite.client.IgniteClient;
import org.apache.ignite.configuration.ClientConfiguration;
public class SimpleCacheApp
{
public static void main(String[] args) {
//Specify the IP and port of the Apache Ignite server to connect to.
ClientConfiguration cfg = new ClientConfiguration().setAddresses("192.168.20.71:10800");
try (IgniteClient igniteClient = Ignition.startClient(cfg)) {
final String CACHE_NAME = "put-get-example";
ClientCache<Integer, String> cache = igniteClient.getOrCreateCache(CACHE_NAME);
Integer key = 1;
String val = "put-get-test";
cache.put(key, val);
System.out.format("PUT [%s]\n", val);
String cachedVal = cache.get(key);
System.out.format("GET [%s]\n", cachedVal);
}
catch (ClientException e) {
e.printStackTrace();
}
catch (Exception e) {
e.printStackTrace();
}
}
}
The execution result is as follows. You can put the data "put-get-test" and confirm that you can get the value from the cache.
PUT [put-get-test]
GET [put-get-test]
To check that it is cached on the server side, try using Ignite's command line interface "Ignite Visor".
First, start Visor.
# /opt/apache-ignite/bin/ignitevisorcmd.sh
The Visor console will be displayed. Connect to the cluster with the "open" command.
visor> open
Local configuration files:
+==========================================================================================+
| # | Configuration File |
+==========================================================================================+
| 0 | config/default-config.xml |
～～ Omitted ～～
A list of configuration files will be displayed, so the default "0" currently used: config/default-config.Select xml.
Choose configuration file number ('c' to cancel) [0]: 0
If you execute the "cache" command here, the cache status will be displayed.
visor> cache
Time of the snapshot: 2018-08-18 03:18:21
+========================================================================================================================+
| Name(@) | Mode | Nodes | Entries (Heap / Off-heap) | Hits | Misses | Reads | Writes |
+========================================================================================================================+
| put-get-example(@c0) | PARTITIONED | 1 | min: 1 (0 / 1) | min: 0 | min: 0 | min: 0 | min: 0 |
| | | | avg: 1.00 (0.00 / 1.00) | avg: 0.00 | avg: 0.00 | avg: 0.00 | avg: 0.00 |
| | | | max: 1 (0 / 1) | max: 0 | max: 0 | max: 0 | max: 0 |
+------------------------------------------------------------------------------------------------------------------------+
You can see that one cache (see Entries) is stored in the "put-get-example" cache.
Make Apache Ignite into a cluster configuration (2 units).
I used to configure one server earlier, but let's set it up with two servers. Create a configuration file (default-config-cluster2server.xml) for a two-unit configuration.
# cp -p /opt/apache-ignite/config/default-config.xml /opt/apache-ignite/config/default-config-cluster2server.xml
Perform cluster settings using the TCP / IP discovery method.
Modify the "default-config-cluster2server.xml" file and add "\
- Addition: In the first place, if nothing is set, it seems to work with multicast by default. If the environment allows multicast, I didn't have to set anything.
Create the same file on two servers. (Or copy with scp)
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<!--
Alter configuration below as needed.
-->
<bean id="grid.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
<property name="addresses">
<list>
<value>igniteserver1:47500..47509</value>
<value>igniteserver2:47500..47509</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
</bean>
</beans>
This time, start two Apache Ignite nodes from Visor. You need to be able to connect with ssh in order to operate the remote node from the Visor. Set with the following command.
$ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
$ chmod 600 ~/.ssh/authorized_keys
Make sure you can set it without a passphrase.
$ ssh igniteserver1
$ ssh igniteserver2
Start the Visor console and execute the following command.
# /opt/apache-ignite/bin/ignitevisorcmd.sh
The Visor console will be displayed. Connect to the cluster with the "open" command.
visor> open
~ Omitted ~
| 6 | config/default-config-cluster2server.xml
~ Omitted ~
Choose configuration file number ('c' to cancel) [0]: 6
+------------------------------------------------------------------------------------+
| Status | Connected |
| Ignite instance name | <default> |
| Config path | /opt/apache-ignite/config/default-config-cluster2server.xml |
| Uptime | 00:00:00 |
+------------------------------------------------------------------------------------+
visor> start -h=192.168.20.71~72 -u=root -k=/root/.ssh/id_rsa -g=/opt/apache-ignite -c=config/default-config-cluster2server.xml
+-------------------------------+
| Successful start attempts | 2 |
| Failed start attempts | 0 |
+-------------------------------+
"Successful start attempts" is "2", which means that the two servers have been successfully started.
To terminate all nodes, execute the following command.
visor> kill -k
Are you sure you want to kill ALL nodes? (y/n) [n]: y
You are about to kill ALL nodes. Are you 100% sure? (y/n) [n]: y
Connect to Ignite cluster from Java program
Nodes in an Ignite cluster include server nodes that store caches and client nodes that do not have data. From a Java program, connect to the Ignite cluster as a client node. The point is to specify the client as "Ignition.setClientMode (true);".
Bring "default-config.xml" from the server and code it as follows: The setting reads "default-config.xml", but it is also possible to configure it programmatically without using the configuration file.
The program created when one unit was configured can connect to multiple units, but when one unit went down, the cause was unknown and failover did not work, so I rewrote it.
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteCache;
import org.apache.ignite.Ignition;
import org.apache.ignite.client.ClientException;
public class SimpleCacheApp2Server
{
public static void main(String[] args) {
Ignition.setClientMode(true);
try (Ignite igniteClient = Ignition.start("default-config-cluster2server.xml")) {
final String CACHE_NAME = "put-get-example";
IgniteCache<Integer, String> cache = igniteClient.getOrCreateCache(CACHE_NAME);
Integer key = 1;
String val = "put-get-test";
cache.put(key, val);
System.out.format("PUT [%s]\n", val);
String cachedVal = cache.get(key);
System.out.format("GET [%s]\n", cachedVal);
}
catch (ClientException e) {
e.printStackTrace();
}
catch (Exception e) {
e.printStackTrace();
}
}
}
reference
--Getting Started --Ignite Official Site -Install Apache Ignite on Ubuntu Linux 16.04 LTS
Recommended Posts