Unilang - The new programming language independently developed by deepin

Deepin has now launched a project of a new programming language, Unilang, aiming at the effective and flexible development of desktop applications. In this article, we will discuss the global design and features of the language to see its highlights.

Unilang is a modularized, highly portable and extensible modern general-purpose programming language. With some revolutionary designs of the fundamental features, it has better abstraction power and flexibility than most contemporary languages.

Importance of modularization

The "modularization" here may be different from what most developers have known. It is mostly for the language itself. In other words, it is not about the code written in the language, but the specification of the language, which always needs manual work to meet new requirements.

For instance, C++ has ... say, n features in total. So, when proposing a new feature as an addition to the language, it is required to consider the interaction with the current n features. If there are some problems, you may have to roll it back, change and re-propose the modified feature. This can be as complex as O(n^2) work, in the notation of algorithmic complexity. Eventually, this work should be done manually. This is not "modular", and inefficient.

For historical reasons, among mainstream languages, C++ performs badly in this aspect. In the last few years, there were indeed several cases of removal of new features before publishing a new version of language standards, and users actually observed the delay. The insufficient modularization and orthogonality in the language design is the key reason for preventing the consensus and resulting in the delay.

Modularization in this sense was often neglected (or not emphasized enough) in the past, because most developers didn't participate in improving the language, so they couldn't easily find the advantages of modularization.

High portability and extensibility

The high portability of Unilang is mainly reflected in the easy reuse of code: there is little work or even no need to modify the code for different environments. The high extensibility is mainly reflected in adding new functions to existing languages by programs without affecting the compatibility of other parts.

High portability

The emphasis on first-class objects in Unilang makes almost all components of source programs are easily reused than ever, as long as the user wants.

Some modern languages (like Scheme and Rust) support hygienic procedure macro, which allows being invoked in the syntax like function calls, without the maintainability defects of the traditional text-based or token-based macro expansion. There are still restrictions on procedure macros over "real" functions (procedures). For example, macro itself is not a first-class object, so it cannot be passed as a function argument. Moreover, implementing a macro is similar to implementing a function body in syntaxes, but not identical. To support the macros, there are usually some specific template sublanguages, which are not completely exchangeable with the syntax used to write **the function body.

In contrast, Unilang does not provide procedural macro, but the code can also have the advantages of procedural macros. The replacement of procedure macro is the (more) general function. The only difference to ordinary ones is they do not evaluate the arguments in the call. As a first-class object, a function can be used as an actual argument of another function, or being returned from a call to another function. There is no additional ad-hoc syntax, so it is easier to learn and use.

More importantly, the call of a Unilang function does not necessarily behave as an instance of macro invocation which needs** to be expanded before the execution of other code. This is equivalent to adding the ability to expand macros at runtime, which provides more powerful abstraction across translation phases and reduces redundancy. In particular, they can be used to implement compilers. As long as the implementer wants, the logic can be directly used as callable code fragments, and their implementation code does not need to be adjusted for different runtime phases. In the traditional language implementation, even if the compiler author wants to call the functions of the compiler at runtime, a complete set of JIT API is required**, and the implementation is often quite different from the AOT compiler, which is difficult to reuse.

High extensibility

The underlying design of the basic language and language extensions of Unilang gives developers the opportunity to smoothly improve the design and implementation of existing language and maintain compatibility in an unprecedented way - rather than waiting for communication and feedback from language designers and implementers.

The current design of Unilang provides the basic language and high-level language. The latter is mainly the library implemented by the former. There are also many features in the basic language that allow the use of such derived implementations, although some of them provide implementations natively due to performance and other reasons (but these are entirely implementation details).

in most cases, not only the language designer can add features to the language to meet the needs of different domains, users can also use the Unilang program to add new features to the existing language by libraries, or hide some features from downstream developers. This process completely meets the requirements of the language specification, without any changes to the language specification and implementations.

A key difference from existing languages is that Unilang can implement a wider range of features through libraries. For example, Unilang does not need built-in support of features such as module management, because these functions can be precisely defined by the library API; Unilang does not need to provide an ad-hoc syntax for creating functions, because this is only a function that can be provided by a constructor that creates functions - keywords such as lambda are ordinary library functions in Unilang (in fact, lambda does not even need to be implemented natively, because it can be derived from other more fundamental functions). This means that if users think that the standard features provided by Unilang do not meet their needs, they have more freedom to customize, still keeping a considerable degree of compatibility with other Unilang programs (instead of creating non-standard dialects of languages).

In terms of smooth language improvement, another existing practice is language-oriented programming (LOP) first advocated by Racket. Racket's main approach is to make the basic language implementation easy to be extended. Language customizers add different designs to get different domain-specific languages (DSLs). DSL can meet some requirements of extension, but it is also easy to cause fragmentation, not easy to reuse code, and has large additional learning and deployment costs. In contrast, the result of Unilang's language extension is still compatible with Unilang by default, providing a new route, namely language derivation , partially equivalent to LOP.

There are traditional examples of extending existing languages, such as the deriving of C++ from C and the deriving of C++/CLI* from C++, which can also be regarded as language derivation. However, this method requires manual extension and maintenance of new language specifications, which is not programmable in principle. Unilang's approach emphasizes programmability and compatibility more than creating new languages (although it is not limited to doing so), so this manual work is not required.

Of course, not all development scenes can take advantage of this approach. But this does not affect the availability of the rest of the language. For common application development scenes, you can directly use the high-level language features. Developers who have needs to customize the language as the **root solution can choose a feature set suitable for their own and extend the language by themselves.

However, to give full play to the advantages here, there is a very practical engineering problem: a serious shortage of manpower to develop the underlying features.

After all, from the functional point of view, whether written as a library or directly placed in a language, the maintainer should** accurately understand the requirements and design interfaces. Therefore, in the short term, we still give priority to solutions that focus on desktop application development.

Remarkable global feature design

Most noteworthy global feature designs in Unilang are:

• Thanks to the explicit evaluation and the design of the first-class environment, the language has homoiconicity. The language does not provide a static type** system natively but uses latent typing instead. As a dynamic language, such design can provide more extensibility at the language level than mainstream languages - for example, allowing users to add and extend the type systems by themselves.
• Global GC is not needed, and a subset of the language allows "unsafe" resource management at the same level as C++, ensuring deterministic resource allocation.
• It supports PTC (proper tail call) without requiring the user to modify the program to prevent undefined behaviors such as stack overflow.
• It has good C++ interoperability.

By the way, technically, the current features of the basic language are provided through several root environments (although this environment is invisible to users), and all the features provided by identifiers are uniformly provided in the basic environment that inherits these root environments.

An invitation from deepin community

Currently, Unilang has provided an implementation of an interpreter that supports REPL and running scripts written in Unilang. We will continue to improve the standard library and other key libraries to improve the developer experience.

At this stage, we plan to support the library based on Qt bindings, to make the work easy in transitioning** some existing desktop application projects. Other framework solutions may be provided in the future. We also plan to provide an execution engine with optimized JIT code generation after the completion of stable basic language features implementation in the interpreter. The new implementation can directly replace the relevant part of the core of the existing interpreter without changing the Unilang code, getting a "free" significant performance improvement.

In addition to the above plans for practical application development, we also encourage other experimental extensions on Unilang to take advantage of the advanced achievements of the programming language community. We expect that these efforts will make Unilang one of the root technologies.

If you are interested in Unilang, welcome to visit the GitHub page of Unilang project to learn about the installation, compilation, and how to participate (details here). Welcome to communicate with us!