Return audio data with api.ai webhook

Overview

• api.ai can return rich messages such as audio data, videos, files, etc. as Custom Payload, not just text.
• Reference) Rich Messages
• It seems that not all integrations are supported
• This is a memorandum of what kind of response should be returned when webhooking with facebook messenger.

Implementation

It's not like *, but if you want to return audio data, for example, we will implement the following json return.

fulfillment

"fulfillment": {
      "messages": [
        {
          "payload": {
            "facebook": {
              "attachment": {
                "payload": {
                  "url": "https://hogehoge/audio/description.mp3"
                },
                "type": "audio"
              }
            }
          },
          "type": 4
        }
      ]
    }

• Controller. If Fulfillmet is returned as it is, an Exception will occur, so it is returned as a character string.

MainController.java

@Controller
public class MainController {
	
	private final MainService mainService;
	
	public MainController(MainService mainService) {
		this.mainService = mainService;
	}
	
	@ResponseBody
	@PostMapping(value="/webhook")
	public String webhook(HttpServletRequest request) {
		return mainService.createFulfillmentJson();
	}
}

• Make json.
• According to the rich message, ResponseSpeech, ResponseImage, ResponsePayload, etc. that inherit from ResponseMessage are prepared respectively.

MainService.java

@Slf4j
@Service
public class MainService {
	private final Gson gson = GsonFactory.getDefaultFactory().getGson();
	
	public String createFulfillmentJson() {
		JsonObject urlJson = new JsonObject();
		urlJson.addProperty("url", "https://hogehoge/audio/description.mp3");
		
		JsonObject payloadJson = new JsonObject();
		payloadJson.add("payload", urlJson);
		payloadJson.addProperty("type", "audio");
		
		JsonObject attachmentJson = new JsonObject();
		attachmentJson.add("attachment", payloadJson);
		
		JsonObject facebookJson   = new JsonObject();
		facebookJson.add("facebook", attachmentJson);
		
		ResponsePayload responsePayload = new ResponsePayload();
		responsePayload.setPayload(facebookJson);
		
		List<ResponseMessage> messages = Collections.singletonList(responsePayload);

		Fulfillment fulfillment = new Fulfillment();
		fulfillment.setMessages(messages);

		return gson.toJson(fulfillment);
	}
}

• Voice data is returned like this.

reference

• I didn't have a good sample, so I referred to the SDK test code. FulfillmentTest.java
• API reference on facebook side Send API Reference (https://developers.facebook.com/docs/messenger-platform/send-api-reference/contenttypes)