# [PYTHON] Implement FReLU with tf.keras

# Overview

I will introduce the implementation of a new activation function called FReLU in keras.

For more information about FReLU + implementation in pytorch, please read the following article.
It is explained very carefully.
Reference: Birth & explanation of new activation function "FReLU"!

This article is about implementing FReLU in tf.keras.
With Depthwise Conv2D and Lambda, you can implement it quickly.

# Definition

```
y= \max(x,\mathbb{T}(x))
```

$ \ mathbb {T} (\ cdot) $ is DepthwiseConv2D.

# Premise

`tensorflow 2.3.0`

# Implementation

`FReLU.py`

```
from tensorflow.keras.layers import DepthwiseConv2D,BatchNormalization
import tensorflow as tf
from tensorflow.keras.layers import Lambda
from tensorflow.keras import backend as K
#Mathematical symbol max()Definition of
def max_unit(args):
inputs , depthconv_output = args
return tf.maximum(inputs, depthconv_output)
def FReLU(inputs, kernel_size = 3):
#T(x)Part of
x = DepthwiseConv2D(kernel_size, strides=(1, 1), padding='same')(inputs)
x = BatchNormalization()(x)
#Calculate tensor shape for Lambda
x_shape = K.int_shape(x)
#max(x, T(x))Part of
x = Lambda(max_unit, output_shape=(x_shape[1], x_shape[2], x_shape[3]))([inputs, x])
return x
```

# At the end

The implementation itself is easy, isn't it?
I will try to implement the Layer version when I have time. It's nice because the name will appear when that person summarizes.
If you have any questions or concerns, please leave a comment.