Consideration when you can do a good job in 10 years with Python3 and Scala3.

Introduction

Roughly speaking, my work that I have been doing for the past few years looks like the following.

python3 series
- numpy+C/C++Build a pipeline that handles large amounts of data with the tools written in.
-In pandas, read RDB and handle excel and csv.

scala2 series
-Build a pipeline to handle large data with Spark.
- Playframework/Maintain a web app made with Akka.

Python probably won't change much for a while, but Scala will gradually move to the 3rd series, which will be available this year. In Scala3 Scala 3, you'll be able to write indent-based syntax like Python! As introduced in, it seems that it will be possible to write python-friendly "mo" as follows (although there seems to be a lot of opposition at the moment).

indent (excerpt).scala

enum Day
  case Monday, Tuesday, Wednesday, Thursday, Friday, Saturday, Sunday
  def isWeekend: Boolean = this match
    case Saturday | Sunday => true
    case _ => false

def fromString(str: String): Day =
  try Day.valueOf(str)
    catch
      case _: IllegalArgumentException =>
        throw new IllegalArgumentException(s"$str is not a valid day")

def use(dayString: String) =
  val day = fromString(dayString)

  if day.isWeekend
    println("Today is a weekend")
    println(s"I rest {dayString}")
  else
    println("Today is a workday")

Source: https://github.com/hinastory/dotty_examples/blob/master/src/main/scala/indent_based/IndentBasedExample.scala

With the support of the IDE, I rarely envy python when writing scala2, but I feel that the indented syntax is more readable.

By the way, people who are working with python3 now think lightly about how happy they will be 10 years after mastering the scala3 system (hereinafter, python3 can also be used as appropriate). Assumption).

Expected work in 10 years

[1] Do microservices work with Scala3.

A microservice that has been a petite puzzle for a long time. Since it contributes to the continuous improvement of surviving IT services, it is expected that the number of jobs to create microservices will increase little by little in the future. In the case of microservices, the implementation language does not matter so much as long as the interface is properly cut. It seems to be determined by the ease of gathering engineers and the high productivity of the language. Scala has been introducing the microservices framework lagom for several years. In recent years, lagom is built on playframwork + akka, which has become a well-established option for web-based companies, and is expected to be used for microservice development in the future. Reference Lagom 1.5, a microservices framework, introduces Akka Management and supports Kubernetes and OpenShift

(Although it is a study level because I have no development experience) In microservices, it is basic to create an asynchronous service that makes full use of Future etc. (Hereafter, I wrote it in scala3 indent syntax style (?)).

HelloStreamServiceImpl_indent.scala

package com.example.hellostream.impl

import com.lightbend.lagom.scaladsl.api.ServiceCall
import com.example.hellostream.api.HelloStreamService
import com.example.hello.api.HelloService

import scala.concurrent.Future

/**
  * Implementation of the HelloStreamService.
  */
class HelloStreamServiceImpl(helloService: HelloService) extends HelloStreamService
  def stream = ServiceCall x =>
      Future.successful(x.mapAsync(8)(helloService.hello(_).invoke()))

Reference Create a new application for the microservice framework Lagom

Considering debugging, it would be safer to write asynchronous processing in scala, which is the compilation language.

As a bonus, write the client side with lagom.js

For a long time, scala.js, which was version 0.6, has reached version 1.0 in 2020. https://www.scala-js.org/

It's not widely used, but there may be an option to write the microservices client side in scala.js 10 years later (although not yet). Looking at the following, it seems that development of lagom.js for microservices clients is in progress. https://github.com/mliarakos/lagom-scalajs-example

It seems that you write using pattern matching on the client side like this.

client_invoke.scala

client.greeting.invoke().onComplete
  case Success(message) => // display message
  case Failure(exception) => // handle exception

[2] Data engineering work with Scala3

Scala2 + Spark2 is one of the standard for big data projects on the cloud such as Amazon EMR and MS databricks. The accumulated data will increase in the next 10 years, so it is natural that Scala3 + Spark3 (?) Will have a job in 10 years. However, there are likely to be more talks that pyspark is fine. I think this is one of my favorite jobs at Scala3, but since there is a lot of information on the web, I will omit the details.

[3] AI engineer work with Scala3

I think that I can eat as an AI engineer even after 10 years with a focus on python, but if there is a lot of learning and it becomes necessary to write complicated distributed processing etc., if you can use the JVM language Scala3 It seems to spread. The technology to be used will be combined with the above-mentioned microservices on Akka and data processing using Spark, so it is abbreviated. ... Personally, I expect that the AI framework on the cloud that can be accessed via the JVM will mature in the next 10 years.

Bonus: How much can Scala3 be used in AI & analysis work for multiplayer game development?

... this is from my personal interest. Along with microservices and data engineers, the gaming industry is a growing industry. In game development, the client will continue to be C / C ++ for the time being, and on the server side, Go, which runs a single binary server, seems to be strong. If you go with a single language, is it C # that has Unity and can go both server side and WebAssembly? From the perspective of the next 10 years, how will rust be involved here?

scala3 is probably not widely used in game development. However, if you want to make full use of the AI engine on the JVM, it may be possible to do something with Python + Scala + α. There is scala-native, but it is unlikely that everything up to the client side will be scala.

As a server-oriented person, I hope that the use of WebAssembly for rendering on the server side and regardless of platform will expand. I wonder if the majority of game development will be done on the server side, and the code on the client side should be left to a craftsman engineer. WebAssembly 1.0 is the fourth language that runs natively on the browser as a W3C recommendation Personally, I'm looking forward to the language vlnag / nim that will be transpiled to C.

At the end

It's not Scala3, but 10 years later, it's still a combination of Python and Go or C #. If not, I wonder if the future of a project manager who can use Python for enterprise DX projects will be safe for the next 10 years. That's it.