Output HTTP header of google-http-client to log

It answers the need to log the HTTP request header / response header issued by the GCP client library.

Postscript: I noticed that there is a description in the head family. The content is almost the same as this memo so far, but I think you should take a look at the original website. https://googleapis.github.io/google-http-java-client/http-transport.html

If you want to log only the request header (do not want to output the body), this article will be helpful. (End of postscript)

Preliminary research

It seems that google-http-client uses JUL (java util logging).

--The Logger used for log output is the LOGGER constant of the com.google.api.client.http.HttpTransport class. .36.0 / google-http-client / src / main / java / com / google / api / client / http / HttpTransport.java # L73). --It seems to be a guideline to refer to the HttpTransport constant from each class instead of having a Logger for each class. --Log level to [CONFIG](https://github.com/googleapis/google-http-java-client/blob/v1.36.0/google-http-client/src/main/java/com/google/api/ If you do client / http / HttpRequest.java # L891), it seems that a log will be output.

From this, I added that if you set the log level to CONFIG for the Logger of HttpTransport, it seems that you can output it.

Try to log

There are two types of logging approaches.

Approach 1: Output in logging.properties

JUL can be set in a file, so let's try that approach first.

Reference: https://docs.oracle.com/javase/jp/8/docs/api/java/util/logging/LogManager.html

Create a configuration file logging.properties.

logging.properties

handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level=CONFIG
com.google.api.client.http.HttpTransport.level=CONFIG

Try running the system properties java.util.logging.config.file with the path to logging.properties (the path on the file system).

This is the code to execute. (Get a list of Cloud Storage buckets)

final GoogleCredentials credentials = createCredentials();
final StorageOptions storageOptions = StorageOptions.newBuilder().setCredentials(credentials).build();
final Storage storage = storageOptions.getService();
final Page<Bucket> buckets = storage.list();
for (final Bucket bucket : buckets.iterateAll()) {
    System.out.println("bucket: " + bucket);
}

Output result

A request / response was output to the console. (It turned out that the preliminary investigation was correct)

July 12, 2020 12:14:09 am com.google.api.client.http.HttpRequest execute
Constitution: -------------- REQUEST  --------------
GET https://storage.googleapis.com/storage/v1/b?project=xxxx&projection=full
Accept-Encoding: gzip
Authorization: <Not Logged>
User-Agent: (Abbreviation)


July 12, 2020 12:14:09 am com.google.api.client.http.HttpRequest execute
Constitution: curl -v --compressed -H 'Accept-Encoding: gzip' -H 'Authorization: <Not Logged>' -H 'User-Agent: (Abbreviation)
July 12, 2020 12:14:10 am com.google.api.client.http.HttpResponse <init>
Constitution: -------------- RESPONSE --------------
HTTP/1.1 200 OK
Alt-Svc: (Abbreviation)
Server: UploadServer
Cache-Control: private, max-age=0, must-revalidate, no-transform
X-GUploader-UploadID: (Abbreviation)
Vary: X-Origin
Vary: Origin
Expires: Sat, 11 Jul 2020 15:14:10 GMT
Content-Length: 3228
Date: Sat, 11 Jul 2020 15:14:10 GMT
Content-Type: application/json; charset=UTF-8


July 12, 2020 12:14:10 am com.google.api.client.util.LoggingByteArrayOutputStream close
Constitution: Total: 3,228 bytes
July 12, 2020 12:14:10 am com.google.api.client.util.LoggingByteArrayOutputStream close
Constitution: {
  "kind": "storage#buckets",
  "items": (Abbreviation)

If you try it this way, you'll notice both what you're aiming for and what you didn't expect.

What was intended

The request and response headers have been successfully output.

What I didn't intend

--Not only the header but also the body was output. --The body is being output from the LoggingByteArrayOutputStream class. --Headers are output from the HttpRequest and HttpResponse classes. --The value of the Authorization header is abbreviated as ʻAuthorization: . --If you change the log level from CONFIG to ALL, the value will be displayed. --The string for issuing the same request with the curl command is output. -(I saw the JUL default log for the first time in a while, but it's hard to read. I want to specify the formatter in logging.properties`)

I want to keep the body out, but I don't think it can be expressed in the configuration file.

Approach 2: Output programmatically

Next, let's try the approach set programmatically.

Instruct JUL's Logger in the same way as the previous configuration file.

import java.util.logging.Handler;
import java.util.logging.Level;
import java.util.logging.LogRecord;
import java.util.logging.Logger;

import com.google.api.client.http.HttpRequest;
import com.google.api.client.http.HttpResponse;
import com.google.api.client.http.HttpTransport;

(Abbreviation)

final Logger logger = Logger.getLogger(HttpTransport.class.getName());
logger.setLevel(Level.CONFIG);
//Return the TODO added Hander(remove)To do
logger.addHandler(new Handler() {
    @Override public void publish(final LogRecord record) {
        //I want to put out only the header this time(I don't want to put out the body)So only HttpRequest and HttpResponse.
        final String sourceClassName = record.getSourceClassName();
        if (HttpRequest.class.getName().equals(sourceClassName) || HttpResponse.class.getName().equals(sourceClassName)) {
            final String date = DateTimeFormatter.ISO_INSTANT.format(Instant.ofEpochMilli(record.getMillis()));
            final String msg = "[" + logger.getLevel() + "] " + date + " " + record.getMessage();
            //Please go to TODO's own logger
            System.out.println(msg);
        }
    }

    @Override public void flush() {
    }

    @Override public void close() throws SecurityException {
    }
});

Output result

When I run the same code (getting a Cloud Storage bucket) as before, This time the body is no longer output. (Although the character string for curl is still output)

[CONFIG] 2020-07-11T13:37:42.844Z -------------- REQUEST  --------------
GET https://storage.googleapis.com/storage/v1/b?project=xxxx&projection=full
Accept-Encoding: gzip
Authorization: <Not Logged>
User-Agent: (Abbreviation)


[CONFIG] 2020-07-11T13:37:42.844Z curl -v --compressed -H 'Accept-Encoding: gzip' -H 'Authorization: <Not Logged>' -H 'User-Agent: (Abbreviation)
[CONFIG] 2020-07-11T13:37:43.536Z -------------- RESPONSE --------------
HTTP/1.1 200 OK
Alt-Svc: (Abbreviation)
Server: UploadServer
Cache-Control: private, max-age=0, must-revalidate, no-transform
X-GUploader-UploadID: (Abbreviation)
Vary: X-Origin
Vary: Origin
Expires: Sat, 11 Jul 2020 13:37:43 GMT
Content-Length: 3228
Date: Sat, 11 Jul 2020 13:37:43 GMT
Content-Type: application/json; charset=UTF-8

Impressions

I tried two different approaches. My use case is

--Case where you want to put out only the header --Case where you want to put out both the header and the body

Since there are two, the approach of outputting in a program seems to be better. (In the code example in the article, only the header is output, but it is not difficult to change it so that both the header and body can be output.)

On the other hand, if you just want to put out both the header and the body, the config file approach is sufficient.

environment

This is the version that has been confirmed to work.

• Windows 10
• Java 8
• https://mvnrepository.com/artifact/com.google.http-client/google-http-client/1.36.0 --It is included in https://mvnrepository.com/artifact/com.google.cloud/google-cloud-storage/1.11.12.