Lark Basics (Python, Lark to make a shell-like guy)

First

Since I am a beginner in programming and a fairly ignorant man, I do not know various names and ambiguous expressions such as "*** na guy" are used, but please forgive me.

Things to do

I want to make a guy (I don't know what to say) that works in a bash-like way of writing! So I made it.

The created program analyzes and executes the following script, for example, and returns the result.

rand -s3 a b c + say def | replace e E

(The result will be, for example, b a cd Ef)

In this article, I will omit the specific processing and briefly explain how to use Lark.

Library you are using Lark - a modern parsing library for Python

Creating a grammar rule file

First, create a text file that describes the grammar rules to be passed to the parser generator. In this case, A script is one or more chunks connected by a join. chunk is one or more sentences connected by pipe ... I will describe it like this.

Grammar.lark

script : [chunk join] chunk

chunk : [(sentence pipe)+] sentence

sentence : command [space option] [(space arg)+] [/\n+/]

command : chars
        | ([chars] subshell [chars])+
option : "-" (chars|subshell)+
arg : chars
    | "'" allchars "'"
    | ([chars] subshell [chars])+

subshell : "(" script ")" [/\n+/]

chars : /[^\+\|\s\(\)']+/[/\n+/]
allchars : /[^']+/[/\n+/]
space : " "
join : [space] "+" [space]
pipe : [space] "|" [space]

Creating a parser

How to create a parser. Just load the text file you just created and pass it to Lark.

Parser creation part

from lark import Lark

with open("Grammar.lark",encoding="utf-8") as grammar:
    LP = Lark(grammar.read(),start="script")

In the second argument start = of Lark (), write the outermost component (probably).

Let's try parsing rand -s3 a b c + say def | replace e E with this parser. You can display it with high readability by using the pretty () method of the analysis result tree, so let's use it.

Parser test

tree = LP.parse("rand -s3 a b c + say def | replace e E")
print(tree.pretty())

result

script
  chunk
    sentence
      command
        chars   ra
      space
      option
        chars   s3
      space
      arg
        chars   a
      space
      arg
        chars   b
      space
      arg
        chars   c
  join
    space
    space
  chunk
    sentence
      command
        chars   sa
      space
      arg
        chars   de
    pipe
      space
      space
    sentence
      command
        chars   re
      space
      arg
        chars   e
      space
      arg
        chars   E

(That? Space is strange? I don't use it for processing ...)

Creating a transformer

Now, let's create a part that executes processing according to the state of the analyzed tree.

Transformer creation part

from lark import Transformer

class mytransformer(Transformer):
    def __init__(self): #Initialize variables to be used as appropriate
        self.var1 = None
        self.var2 = []

    def sentence(self,tree): #If you take the second argument, you can use a tree deeper than sentence!
        print("sentence") # example

    def command(self,tree):
        print("command")

    def option(self,tree):
        print("option")

Processing such as ʻargandchunkis omitted. Sentence, command, ʻoption, etc. defined in mytransformer are called ** with depth-first priority. So in this example The processing is performed in the order of command-> ʻoption-> sentence`.

How to use the transformer

How to use the transformer

tree = LP.parse("rand -s3 a b c + say def | replace e E")
mytransformer().transform(tree)

Use it like this.

Finally

So far, I have briefly explained about Lark.

Since the beginners managed to do their best, I think there are some mistakes. If you find any mistakes, please leave a comment.