Understand bugs by implementing and analyzing them (1) Deadlock (Java)
Overview
There are many ways to analyze problems such as memory leaks and deadlocks on the Internet. However, since it is not known when and what kind of problem will appear in front of you, you will systematically learn and practice the analysis method before you actually face the problem. I think it's quite difficult to keep it.
Therefore, we developed a bug-filled web application that can intentionally reproduce the problem so that we can learn in a practical format, and also [it] to Qiita. Information about](http://qiita.com/tamura__246/items/7275c7406706fb0057e8) has been released. This web application is very easy to build and start with a single command, so I think it can be used for experimental and educational purposes as well.
Using this web application, I would like to introduce various problems (bugs) and how to analyze them little by little.
So, the first bug I'll introduce is a Java thread ** deadlock **.
Reproduce deadlock
First, let's actually reproduce the deadlock. Download and launch the Web Application.
$ java -jar easybuggy.jar
Or
$ git clone https://github.com/k-tamura/easybuggy
$ cd easybuggy
$ mvn clean install exec:exec
You can also start it by deploying ROOT.war to a servlet container such as Tomcat. When the following message is displayed, the start of the Web application is complete.
5 27, 2017 3:29:58 pm org.apache.coyote.AbstractProtocol start
information: Starting ProtocolHandler ["http-bio-8080"]
- Supplement: If the version is 1.2.x or earlier, easybuggy.jar may not work properly on Windows. In that case, set the path to the bin of the JDK in path, and then try starting with the java of the JRE included in the JDK. [^ 1]
> set path=%path%;C:\Program Files\Java\jdk1.8.0_121\bin
> "C:\Program Files\Java\jdk1.8.0_121\jre\bin\java" -jar easybuggy.jar
After starting, access the following URL.
http://localhost:8080
If it starts normally, the following screen will be displayed.
Click on the link that says "Deadlock (Java)" in it. Then you will see a screen that just says "Loading this page several times will cause a deadlock." First, try loading the screen by pressing the F5 key only once. After about 5 seconds, a response will be returned and the message "No deadlock has occurred" will be displayed. Deadlocks do not occur in situations where only a single request is being processed. Then press the F5 key twice in a row. This time, no matter how long you wait, you shouldn't get a response. At this time, the web application has a deadlock.
To analyze
If no response is returned, the following possibilities are possible.
-Processing is stopped (caused by deadlock, etc.) -Processing is not finished (due to infinite loop etc.) -Processing takes longer than expected (due to logic that lacks consideration for performance)
If you don't get a response, the first thing you should do is ** get a thread dump **.
First, specify the process ID of the Web application with the jps command.
＄ jps
85489 org.eclipse.equinox.launcher_1.3.100.v20150511-1540.jar
45733 Launcher
69970 easybuggy.jar
89901 Jps
The process name of this web application is "easybuggy.jar" when it is started by the java command and "EmbeddedJettyServer" when it is started by the mvn command. In the above case, 69970 would be the process ID.
When you execute the jstack command with the specified process ID specified, a thread dump is output.
$ jstack 69970
2017-02-26 18:58:57
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.111-b14 mixed mode):
"Attach Listener" #71 daemon prio=9 os_prio=0 tid=0x00007f4804001800 nid=0x1120f runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"derby.rawStoreDaemon" #70 daemon prio=5 os_prio=0 tid=0x00007f4798deb800 nid=0x111b2 in Object.wait() [0x00007f47e0473000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.apache.derby.impl.services.daemon.BasicDaemon.rest(Unknown Source)
- locked <0x00000000f7aa3a80> (a org.apache.derby.impl.services.daemon.BasicDaemon)
at org.apache.derby.impl.services.daemon.BasicDaemon.run(Unknown Source)
at java.lang.Thread.run(Thread.java:745)
・ ・ ・(Abbreviation)・ ・ ・
Found one Java-level deadlock:
=============================
"http-listener(8)":
waiting to lock monitor 0x00007f4830363368 (object 0x00000000f0165dc8, a java.lang.Object),
which is held by "http-listener(9)"
"http-listener(9)":
waiting to lock monitor 0x00007f4830364b78 (object 0x00000000f0165db8, a java.lang.Object),
which is held by "http-listener(8)"
Java stack information for the threads listed above:
===================================================
"http-listener(8)":
at org.t246osslab.easybuggy.troubles.DeadlockServlet.lock12(DeadlockServlet.java:61)
- waiting to lock <0x00000000f0165dc8> (a java.lang.Object)
-
- locked <0x00000000f0165db8> (a java.lang.Object)
at org.t246osslab.easybuggy.troubles.DeadlockServlet.doGet(DeadlockServlet.java:43)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
・ ・ ・(Abbreviation)・ ・ ・
"http-listener(9)":
at org.t246osslab.easybuggy.troubles.DeadlockServlet.lock21(DeadlockServlet.java:70)
- waiting to lock <0x00000000f0165db8> (a java.lang.Object)
- locked <0x00000000f0165dc8> (a java.lang.Object)
at org.t246osslab.easybuggy.troubles.DeadlockServlet.doGet(DeadlockServlet.java:46)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
・ ・ ・(Abbreviation)・ ・ ・
Found 1 deadlock.
You can see that a deadlock has occurred, as it says "Found one Java-level deadlock:". Deadlocks can be easily detected by thread dumps and can also identify which thread is causing the problem. The above indicates that there is a deadlock between the two threads "http-listener (8)" and "http-listener (9)". You can see that the former is waiting to lock on line 61 of DeadlockServlet.java and the latter is on line 70 of DeadlockServlet.java.
What is deadlock?
What kind of event is deadlock in the first place? Deadlock means that multiple processes (threads in this case) wait for the release of resources (objects in this case) that are occupied (locked) by each other, and the processing stops.
For example, after thread 1 locks object 1 and thread 2 locks object 2, both are not unlocked, but thread 1 locks object 2 and thread 2 locks object 1 as shown below. If you try, both threads will wait for each other to complete and the process will stop. This state is a deadlock.
What kind of implementation is it?
What kind of implementation was the source code? DeadlockServlet.java that was output to the thread dump earlier Let's look at. The important parts are:
private final Object lock1 = new Object();
private final Object lock2 = new Object();
private boolean switchFlag = true;
protected void doGet(HttpServletRequest req, HttpServletResponse res) throws ServletException, IOException {
・ ・ ・(Abbreviation)・ ・ ・
switchFlag = !switchFlag;
if (switchFlag) {
lock12();
} else {
lock21();
}
・ ・ ・(Abbreviation)・ ・ ・
}
private void lock12() {
synchronized (lock1) {
sleep();
synchronized (lock2) {
sleep();
}
}
}
private void lock21() {
synchronized (lock2) {
sleep();
synchronized (lock1) {
sleep();
}
}
}
In this Servlet, a flag called switchFlag alternates requests between the two methods lock12 () and lock21 (). In lock12 (), the object "lock1" is locked → sleep for 5 seconds → the object "lock2" is locked in this order. In lock21 (), processing is executed in the reverse order.
Therefore, this case is as follows.
Two HTTP listener threads (http-listener (8) and http-listener (9)) that were spawned to accept requests from clients wait for their respective locked objects (lock1, lock2) to be released. That is why it is a deadlock.
Detection methods other than thread dump
There are other ways to detect deadlocks.
VisualVM
You can also detect deadlocks using the Visual VM that comes with Oracle's JDK. VisualVM automatically detects deadlocks and displays a warning in red.
Java Mission Control
If you use Java Mission Control, which also comes with Oracle's JDK, you can detect it by checking the "Deadlock detection" checkbox.
JConsole
OpenJDK does not have Visual VM or Java Mission Control, but it does have JConsole. You can also detect this by clicking the "Detect Deadlock" button.
Can the JVM detect a deadlock and terminate one thread?
In a typical RDBMS, when a deadlock is detected, one transaction can be rolled back and the other can be committed. Can the JVM do the same? The answer is "** no **". There is no such JVM option. The only way to ** restore a web application from a deadlock is to reboot **. In general, deadlocks rarely surface immediately, so it can happen that a large number of threads are waiting. Therefore, when implementing the process of locking an object, ** programmers should be careful not to cause inconsistencies in the process order **.
Programmatically detect deadlock
You can programmatically detect deadlocks. You can use java.lang.management.ThreadMXBean.findDeadlockedThreads () to get information about deadlocked threads as shown below.
ThreadMXBean bean = ManagementFactory.getThreadMXBean();
long[] threadIds = bean.findDeadlockedThreads();
if (threadIds != null) {
ThreadInfo[] infos = bean.getThreadInfo(threadIds);
for (ThreadInfo info : infos) {
log.error(info.toString());
}
}
This summary
- Deadlock means that multiple processes (threads) wait for each other's locked resources to be released, and the process stops.
- When a deadlock occurs, the process (thread) is stopped, so in the case of a Web application, the response will not be returned forever.
- Deadlocks can be detected with tools that come with the JDK, such as jstack and JConsole.
- No JVM option to recover from deadlock
- The only way to recover from a deadlock is to restart the JVM
- It is the programmer's responsibility to avoid deadlocks
- Deadlocks can also be detected programmatically (ThreadMXBean.findDeadlockedThreads ())
reference: COMP 3400 Lecture 7: Deadlock (Lecture material at Ottervine University?) http://faculty.otterbein.edu/PSanderson/COMP3400/notes/lecture07.html
[^ 1]: Probably due to a bug in Payara Micro, but the cause has not been identified. I will look it up when I have time.
Recommended Posts