Corresponds to 15 strings

Introduction

In the previous article, Supports function expressions. With the correspondence of function expressions, the type handled as a value is only ʻInteger`, and the function expression is also treated as a value. In the same way, we will handle strings as values.

What you want to do with string support

Check what you want to do with string support. For example, there is the following program. A string enclosed in " is a string literal. When a character string is treated as a logical value under a condition such as an if statement, a character string of length 0 is false. Strings can be concatenated with +. The println () method call at the end of the program aims to output Hello world!.

var object = ""
if (!object) {
 object = "world"
}
println("Hello " + object + "!")

How to implement

We will consider how to implement it in the order of lexical analysis (Lexer), parser (Parser), and interpreter (Interpreter).

How to implement lexical analysis (Lexer)

Since there is no function to parse the string literal enclosed in " , add it.

How to implement parsing

String literals can be treated in the same way as numeric literals, so we'll add definitions according to numeric literals.

How to implement the Interpreter

We will make it possible to handle String as a value in the same way as ʻInteger. The processing of unary and binary operators, which only considered ʻInteger, Change it so that it can also handle String.

Try to implement in Java

Move on to implementation. About lexical analysis (Lexer), parser (Parser), interpreter (Interpreter) Let's take a look at the changes and additions in order.

Lexer.java

An implementation of Lexer.java.

Adds the ability to parse strings enclosed in " .

Add the ʻisStringStart ()method. " `Detects the start of the enclosure.

Lexer.java


    private boolean isStringStart(char c) {
        return c == '"';
    }

Add the string () method. " Paruses up to the end of the box into one token.

Implement by referring to the JSON string below. http://json.org/json-ja.html The reference implementation of 4 Hexadecimal digits has been truncated.

The kind that represents a string literal should be string.

Lexer.java


    private Token string() throws Exception {
        StringBuilder b = new StringBuilder();
        next();
        while (c() != '"') {
            if (c() != '\\') {
                b.append(next());
            } else {
                next();
                char c = c();
                if (c == '"') {
                    b.append('"');
                    next();
                } else if (c == '\\') {
                    b.append('\\');
                    next();
                } else if (c == '/') {
                    b.append('/');
                    next();
                } else if (c == 'b') {
                    b.append('\b');
                    next();
                } else if (c == 'f') {
                    b.append('\f');
                    next();
                } else if (c == 'n') {
                    b.append('\n');
                    next();
                } else if (c == 'r') {
                    b.append('\r');
                    next();
                } else if (c == 't') {
                    b.append('\t');
                    next();
                } else {
                    throw new Exception("string error");
                }
            }
        }
        next();
        Token t = new Token();
        t.kind = "string";
        t.value = b.toString();
        return t;
    }

Modify the nextToken () method. Add the call to the added method to // Add. You can now decompose it into a token of a string literal.

Lexer.java


    public Token nextToken() throws Exception {
        skipSpace();
        if (isEOT()) {
            return null;
        } else if (isSignStart(c())) {
            return sign();
        } else if (isDigitStart(c())) {
            return digit();
            // Add
        } else if (isStringStart(c())) {
            return string();
        } else if (isIdentStart(c())) {
            return ident();
        } else if (isParenStart(c())) {
            return paren();
        } else if (isCurlyStart(c())) {
            return curly();
        } else if (isSymbolStart(c())) {
            return symbol();
        } else {
            throw new Exception("Not a character for tokens");
        }
    }

Parser.java

An implementation of Parser.java.

Add a definition of how it works for the meaning of the token. Added " string " to // Update. String literals, like numeric literals and variable names, Since the syntax cannot be further decomposed, it is added to the factor classification. The factor parsing has already been implemented, so this is the end of the parser implementation.

Parser.java


    public Parser() {
        degrees = new HashMap<>();
        degrees.put("(", 80);
        degrees.put("*", 60);
        degrees.put("/", 60);
        degrees.put("+", 50);
        degrees.put("-", 50);
        degrees.put("==", 40);
        degrees.put("!=", 40);
        degrees.put("<", 40);
        degrees.put("<=", 40);
        degrees.put(">", 40);
        degrees.put(">=", 40);
        degrees.put("&&", 30);
        degrees.put("||", 30);
        degrees.put("=", 10);
        // Update
        factorKinds = Arrays.asList(new String[] { "digit", "ident", "string" });
        binaryKinds = Arrays.asList(new String[] { "sign" });
        rightAssocs = Arrays.asList(new String[] { "=" });
        unaryOperators = Arrays.asList(new String[] { "+", "-", "!" });
        reserved = Arrays.asList(new String[] { "function", "return", "if", "else", "while", "break", "var" });
    }

Interpreter.java

An implementation of Interpreter.java.

ʻExpression () method change. It is a process that branches according to the meaning (kind) of the token that represents the expression. Added a branch under // Add for the token string, which represents a string literal. Processing a string token simply returns its value`.

Interpreter.java


    public Object expression(Token expr) throws Exception {
        if (expr.kind.equals("digit")) {
            return digit(expr);
            // Add
        } else if (expr.kind.equals("string")) {
            return string(expr);
        } else if (expr.kind.equals("ident")) {
            return ident(expr);
        } else if (expr.kind.equals("func")) {
            return func(expr);
        } else if (expr.kind.equals("fexpr")) {
            return fexpr(expr);
        } else if (expr.kind.equals("paren")) {
            return invoke(expr);
        } else if (expr.kind.equals("sign") && expr.value.equals("=")) {
            return assign(expr);
        } else if (expr.kind.equals("unary")) {
            return unaryCalc(expr);
        } else if (expr.kind.equals("sign")) {
            return calc(expr);
        } else {
            throw new Exception("Expression error");
        }
    }

    public String string(Token token) {
        return token.value;
    }

This is a modification of the value () method. The value () method returns the value if the argument is the value itself, such as 1. If it is a variable, it returns the value that the variable holds. It also corresponded when the argument was String.

Interpreter.java


    public Object value(Object value) throws Exception {
        if (value instanceof Integer) {
            return value;
        } else if (value instanceof String) {
            return value;
        } else if (value instanceof Func) {
            return value;
        } else if (value instanceof Variable) {
            Variable v = (Variable) value;
            return value(v.value);
        }
        throw new Exception("right value error");
    }

ʻInteger () method change. The ʻinteger () method guarantees that the argument is ʻInteger. Changed to implicitly try to convert to ʻInteger if the argument was String.

Interpreter.java


    public Integer integer(Object value) throws Exception {
        if (value instanceof Integer) {
            return (Integer) value;
        } else if (value instanceof String) {
            return Integer.decode((String) value);
        } else if (value instanceof Variable) {
            Variable v = (Variable) value;
            return integer(v.value);
        }
        throw new Exception("right value error");
    }

Added the string () method. The string () method guarantees that the argument is String. Implicitly convert to String if the argument is ʻInteger`.

Interpreter.java


    public String string(Object value) throws Exception {
        if (value instanceof String) {
            return (String) value;
        } else if (value instanceof Integer) {
            return value.toString();
        } else if (value instanceof Variable) {
            Variable v = (Variable) value;
            return string(v.value);
        }
        throw new Exception("right value error");
    }

ʻUnaryCalc () method change. The ʻunaryCalc () method is a method that handles unary operators. Before the change, only ʻIntegerwas supported. In order to correspond toString, depending on the result of calling the value () method If it is ʻInteger, call the ʻunaryCalcInteger () method that handles the unary operator for ʻInteger. If it is String, call the ʻunaryCalcString ()method that handles the unary operator forString`.

Interpreter.java


    public Object unaryCalc(Token expr) throws Exception {
        Object value = value(expression(expr.left));
        if (value instanceof Integer) {
            return unaryCalcInteger(expr.value, (Integer) value);
        } else if (value instanceof String) {
            return unaryCalcString(expr.value, (String) value);
        } else {
            throw new Exception("unaryCalc error");
        }
    }

    public Object unaryCalcInteger(String sign, Integer left) throws Exception {
        if (sign.equals("+")) {
            return left;
        } else if (sign.equals("-")) {
            return -left;
        } else if (sign.equals("!")) {
            return toInteger(!isTrue(left));
        } else {
            throw new Exception("unaryCalcInteger error");
        }
    }

    public Object unaryCalcString(String sign, String left) throws Exception {
        if (sign.equals("!")) {
            return toInteger(!isTrue(left));
        } else {
            throw new Exception("unaryCalcString error");
        }
    }

This is a modification of the calc () method. The calc () method is a method that handles binary operators. Before the change, only ʻIntegerwas supported. In order to correspond toString, depending on the result of calling the value () method on the left side, If it is ʻInteger, call the calcInteger () method that handles the binary operator for ʻInteger. If it is String, call the calcString () method that handles the binary operator for String`. The right-hand side may be implicitly converted.

Interpreter.java


    public Object calc(Token expr) throws Exception {
        Object left = value(expression(expr.left));
        Object right = value(expression(expr.right));
        Integer ileft = null;
        String sleft = null;

        if (left instanceof Integer) {
            ileft = (Integer) left;
        } else if (left instanceof String) {
            sleft = (String) left;
        }

        if (ileft != null) {
            return calcInteger(expr.value, ileft, right);
        } else if (sleft != null) {
            return calcString(expr.value, sleft, right);
        } else {
            throw new Exception("calc error");
        }
    }

    public Object calcInteger(String sign, Integer left, Object right) throws Exception {
        if (sign.equals("+")) {
            return left + integer(right);
        } else if (sign.equals("-")) {
            return left - integer(right);
        } else if (sign.equals("*")) {
            return left * integer(right);
        } else if (sign.equals("/")) {
            return left / integer(right);
        } else if (sign.equals("==")) {
            return toInteger(left == integer(right));
        } else if (sign.equals("!=")) {
            return toInteger(left != integer(right));
        } else if (sign.equals("<")) {
            return toInteger(left < integer(right));
        } else if (sign.equals("<=")) {
            return toInteger(left <= integer(right));
        } else if (sign.equals(">")) {
            return toInteger(left > integer(right));
        } else if (sign.equals(">=")) {
            return toInteger(left >= integer(right));
        } else if (sign.equals("&&")) {
            if (!isTrue(left)) {
                return left;
            }
            return right;
        } else if (sign.equals("||")) {
            if (isTrue(left)) {
                return left;
            }
            return right;
        } else {
            throw new Exception("calcIteger error");
        }
    }

    public Object calcString(String sign, String left, Object right) throws Exception {
        if (sign.equals("+")) {
            return left + string(right);
        } else if (sign.equals("==")) {
            return toInteger(left.equals(string(right)));
        } else if (sign.equals("!=")) {
            return toInteger(!left.equals(string(right)));
        } else if (sign.equals("<")) {
            return toInteger(left.compareTo(string(right)) < 0);
        } else if (sign.equals("<=")) {
            return toInteger(left.compareTo(string(right)) <= 0);
        } else if (sign.equals(">")) {
            return toInteger(left.compareTo(string(right)) > 0);
        } else if (sign.equals(">=")) {
            return toInteger(left.compareTo(string(right)) >= 0);
        } else if (sign.equals("&&")) {
            if (!isTrue(left)) {
                return left;
            }
            return right;
        } else if (sign.equals("||")) {
            if (isTrue(left)) {
                return left;
            }
            return right;
        } else {
            throw new Exception("calcString error");
        }
    }

ʻIsTrue ()Method change. Changed so thatString` can be processed as a logical value. Changed to judge a character string of length 0 as false.

Interpreter.java


    public boolean isTrue(Object value) throws Exception {
        if (value instanceof Integer) {
            return 0 != ((Integer) value);
        } else if (value instanceof String) {
            return !"".equals(value);
        } else if (value instanceof Func) {
            return true;
        } else {
            return false;
        }
    }

The program below using the above implementation

var object = ""
if (!object) {
 object = "world"
}
println("Hello " + object + "!")

And The println () method call at the end of the program prints Hello world!.

Interpreter.java


    public static void main(String[] args) throws Exception {
        String text = "";
        text += "var object = \"\"";
        text += "if (!object) {";
        text += " object = \"world\"";
        text += "}";
        text += "println(\"Hello \" + object + \"!\")";
        List<Token> tokens = new Lexer().init(text).tokenize();
        List<Token> blk = new Parser().init(tokens).block();
        new Interpreter().init(blk).run();
        // --> Hello world!
    }

That's all for the implementation. Thank you very much.

in conclusion

The full source is available here.

Calc https://github.com/quwahara/Calc/tree/article-15-string-r3/Calc/src/main/java

There is a continuation article.

** Corresponds to method invocation ** http://qiita.com/quwahara/items/f1bddefe984c8d233e02

Recommended Posts

Corresponds to 15 strings
Corresponds to 17 arrays
Corresponds to Scope
8 Corresponds to multiple arguments
10 Corresponds to if statement
14 Corresponds to function expressions
5 Corresponds to prioritizing parentheses
19 Corresponds to object creation
16 Corresponds to method invocation
How to concatenate strings
9 Corresponds to the return value
12 Corresponds to the while statement
18 Corresponds to JSON-like object definitions
20 Corresponds to static method invocation
to_ ○
11 Corresponds to comparison and logical operators
[Swift] How to replace multiple strings
How to concatenate strings in java
Convert an array of strings to numbers
Association (1 to 1)! !!
[java] Summary of how to handle character strings