18 Corresponds to JSON-like object definitions

Introduction

I would like to support JSON-like object definitions. This article is a continuation of "Corresponding to Array".

What you want to do with JSON-like object definition support

Make sure you want to do it with a JSON-like object definition. For example, there is the following program. Create an object on the first line and assign it to the variable ʻobj. The 5th line outputs the number of object items 2. In the 6th line, we aim to output the value corresponding to the object's key2, "value2" `.

var obj = {
  "key1": "value1",
  "key2": "value2",
}
println(obj.size())
println(obj["key2"])

It does not support access to values by . such as ʻobj.key2`.

How to implement

We will consider how to implement it in the order of lexical analysis (Lexer), parser (Parser), and interpreter (Interpreter).

How to implement lexical analysis (Lexer)

Since there is no function to parse :, add it.

How to implement parsing

The implementation in parsing corresponds to the syntax that generates the object definition. The syntax for accessing the elements of an object is covered by the corresponding syntax for accessing the previous array.

The syntax for creating an object starts with a { token. Implement it in the lead () method, just as the other first token is syntactically determined.

How to implement the Interpreter

I decided to use LinkedHashMap <String, Object> as the object entity.

The processing for the syntax implements the creation of objects and access to object elements. In object generation, LinkedHashMap <String, Object> is generated and elements are added. To access the object element, call the LinkedHashMap <String, Object> :: get () method.

Try to implement in Java

Move on to implementation. About lexical analysis (Lexer), parser (Parser), interpreter (Interpreter) Let's take a look at the changes and additions in order.

Lexer.java

An implementation of Lexer.java.

Adds a lexical analysis function for :.

ʻI change the SymbolStart ()method. Added detection of:`.

Lexer.java


    private boolean isSymbolStart(char c) {
        // Update
        return c == ',' || c == ':';
    }

That's all for changing Lexer.java.

Parser.java

An implementation of Parser.java.

Implements parsing of object creation syntax. Of the lead () method that implements the syntax determined by the first token Added a call to the newMap () method that performs object creation parsing to // Add.

Parser.java


    private Token lead(Token token) throws Exception {
        if (token.kind.equals("ident") && token.value.equals("function")) {
            return func(token);
        } else if (token.kind.equals("ident") && token.value.equals("return")) {
            token.kind = "ret";
            if (!token().kind.equals("eob")) {
                token.left = expression(0);
            }
            return token;
        } else if (token.kind.equals("ident") && token.value.equals("if")) {
            return if_(token);
        } else if (token.kind.equals("ident") && token.value.equals("while")) {
            return while_(token);
        } else if (token.kind.equals("ident") && token.value.equals("break")) {
            token.kind = "brk";
            return token;
        } else if (token.kind.equals("ident") && token.value.equals("var")) {
            return var(token);
        } else if (factorKinds.contains(token.kind)) {
            return token;
        } else if (unaryOperators.contains(token.value)) {
            token.kind = "unary";
            token.left = expression(70);
            return token;
        } else if (token.kind.equals("paren") && token.value.equals("(")) {
            Token expr = expression(0);
            consume(")");
            return expr;
        } else if (token.kind.equals("bracket") && token.value.equals("[")) {
            return newArray(token);
            // Add
        } else if (token.kind.equals("curly") && token.value.equals("{")) {
            return newMap(token);
        } else {
            throw new Exception("The token cannot place there.");
        }
    }

Added newMap () method to perform object creation parsing. Collect the elements separated by , into the params of the { token until the } token is reached. The element holds the key item in the left of the": "token and the value item in the right. Add the ":" token to the params of the {token. The token type kind is set to newMap. If the object is empty, like [{"key1 ":" value1 "}, {" key2 ":" value2 "},] I tried to ignore it as an element.

Parser.java


    private Token newMap(Token token) throws Exception {
        token.kind = "newMap";
        token.params = new ArrayList<Token>();
        while(true) {
            if (token().value.equals("}")) {
                consume("}");
                break;
            }
            Token key = expression(0);
            Token colon = consume(":");
            colon.left = key;
            token.params.add(colon);
            if (token().value.equals(",")) {
                colon.right = blank;
                consume(",");
                continue;
            }
            colon.right = expression(0);
            if (token().value.equals(",")) {
                consume(",");
                continue;
            } else {
                consume("}");
                break;
            }
        }
        return token;
    }

That's all for changing Parser.java.

Interpreter.java

An implementation of Interpreter.java.

ʻExpression () method change. It is a process that branches according to the meaning (kind) of the token that represents the expression. Added a newMap ()method call for object creation under// Add. Changed the ʻaccessArray () method for array access under // Update to the ʻaccessArrayOrMap ()` method. That's because the method now has both array access and object access.

Interpreter.java


    public Object expression(Token expr) throws Exception {
        if (expr.kind.equals("digit")) {
            return digit(expr);
        } else if (expr.kind.equals("string")) {
            return string(expr);
        } else if (expr.kind.equals("ident")) {
            return ident(expr);
        } else if (expr.kind.equals("blank")) {
            return blank(expr);
            // Add
        } else if (expr.kind.equals("newMap")) {
            return newMap(expr);
        } else if (expr.kind.equals("newArray")) {
            return newArray(expr);
            // Update
        } else if (expr.kind.equals("bracket")) {
            return accessArrayOrMap(expr);
        } else if (expr.kind.equals("func")) {
            return func(expr);
        } else if (expr.kind.equals("fexpr")) {
            return fexpr(expr);
        } else if (expr.kind.equals("paren")) {
            return invoke(expr);
        } else if (expr.kind.equals("sign") && expr.value.equals("=")) {
            return assign(expr);
        } else if (expr.kind.equals("unary")) {
            return unaryCalc(expr);
        } else if (expr.kind.equals("sign")) {
            return calc(expr);
        } else if (expr.kind.equals("dot")) {
            return dot(expr);
        } else {
            throw new Exception("Expression error");
        }
    }

Added newMap () method for object creation. Generates LinkedHashMap <String, Object>, adds an element, and returns it.

Interpreter.java


    public Object newMap(Token expr) throws Exception {
        Map<String, Object> m = new LinkedHashMap<>();
        for (Token item : expr.params) {
            String key = identOrString(item.left);
            Object value = value(expression(item.right));
            m.put(key, value);
        }
        return m;
    }

Added a ʻidentOrString ()method that guarantees that the argument token is valid as the key of the object. Used in thenewMap ()method above. If the argument token itself is an identifier or the result of executing the argument token isString`, it is valid.

Interpreter.java


    public String identOrString(Token expr) throws Exception {
        if (expr.kind.equals("ident")) {
            return expr.value;
        } else {
            return string(expression(expr));
        }
    }

Changed the value () method of the method that guarantees that the argument is a value. The first // Add now allowsMap <String, Object>as a value.

Interpreter.java


    public Object value(Object value) throws Exception {
        if (value instanceof Integer) {
            return value;
        } else if (value instanceof String) {
            return value;
        } else if (value instanceof List<?>) {
            return value;
            // Add
        } else if (value instanceof Map<?, ?>) {
            return value;
        } else if (value == null) {
            return value;
        } else if (value instanceof Func) {
            return value;
        } else if (value instanceof Variable) {
            Variable v = (Variable) value;
            return value(v.value);
        }
        throw new Exception("right value error");
    }

ʻAccessArray () method for array access, Changed to ʻaccessArrayOrMap () method for array or object access. The argument ʻexpr is passed a [token. If the[tokenleft is List `, It is treated as an array, and if it is` Map , It is treated as an object. The [tokenright` is treated as an index if it is an array and as a key if it is an object.

Interpreter.java


    @SuppressWarnings("unchecked")
    public Object accessArrayOrMap(Token expr) throws Exception {
        Object v = value(expression(expr.left));
        if (v instanceof List<?>) {
            List<Object> ar = (List<Object>) v;
            Integer index = integer(expression(expr.right));
            return ar.get(index);
        } else if (v instanceof Map<?, ?>) {
            Map<String, Object> map = (Map<String, Object>) v;
            String key = string(expression(expr.right));
            return map.get(key);
        } else {
            throw new Exception("accessArrayOrMap error");
        }
    }

The program below using the above implementation

var obj = {
  "key1": "value1",
  "key2": "value2",
}
println(obj.size())
println(obj["key2"])

And prints the value " v2 " associated with the number of elements 2 and key2 of the object.

Interpreter.java


    public static void main(String[] args) throws Exception {
        String text = "";
        text += "var m = {";
        text += "  key1: \"v1\",";
        text += "  key2: \"v2\",";
        text += "}";
        text += "println(m.size())";
        text += "println(m[\"key2\"])";
        List<Token> tokens = new Lexer().init(text).tokenize();
        List<Token> blk = new Parser().init(tokens).block();
        new Interpreter().init(blk).run();
        // --> 2
        // --> v2
    }

That's all for the implementation. Thank you very much.

in conclusion

The full source is available here.

Calc https://github.com/quwahara/Calc/tree/article-18-map/Calc/src/main/java

There is a continuation article.

** Corresponds to object creation ** http://qiita.com/quwahara/items/ced0e9e2bf487f4021f8

Recommended Posts

18 Corresponds to JSON-like object definitions
19 Corresponds to object creation
Corresponds to 17 arrays
Corresponds to Scope
Corresponds to 15 strings
8 Corresponds to multiple arguments
10 Corresponds to if statement
14 Corresponds to function expressions
5 Corresponds to prioritizing parentheses
16 Corresponds to method invocation
9 Corresponds to the return value
12 Corresponds to the while statement
Convert Serializable Object to byte []
20 Corresponds to static method invocation
11 Corresponds to comparison and logical operators
Convert ruby object to JSON format