Tuesday, June 7, 2016

The Application as a Meta Programming Language


One of Qore's fundamental design goals is to provide a platform that allows programmers to dramatically increase productivity (reduce development time) while producing the high-quality results.

In a given complex functional domain, one approach for achieving this goal is to implement an application framework whose configuration and execution can also be influenced/governed by embedding appropriately-sandboxed user-defined code in the application itself.

Example Telecommunications Billing Application

Let's assume an application that implements some complex functionality, like a billing system for telecommunications services, for example.  With the approach described here, the application allows itself to be extended by supporting embedded and sandboxed user-defined code as attributes of configuration objects which are executed by the application itself in the appropriate context to provide for a very high level of functional customization.

In this hypothetical telecommunications billing system, this user-defined code might by attached to a tariff or discount, for example, and the billing system would then execute the code when calculating fees for consumed services.

The billing system would supply an internal API that could be used by embedded code for monetary calculations and for accessing contextual information, such as the customer's configuration, service usage data, etc.   Furthermore, fees and discounts calculated would be subject to taxation, and the calculation and application of taxation rules could similarly support runtime extensions through code attributes.

To allow for the highest level of user productivity and functional flexibility, our billing system should support simple and standard billing actions through (pure, non-code) configuration while allowing for more complex behavior to be defined through the execution of code attributes on appropriate configuration objects.

Vertical Support in Qore

Qore supports this sort of application development through its sandboxed code / logic containers, which allows applications such as the above to be developed that can be extended and customized with user-defined logic also written in Qore.  These logic containers can be created and destroyed at runtime, which allows for the behavior of the system to be updated without having to restart the server itself.

"Vertical" in this sense refers to the fact that both the server and the embedded code are written in the same programming language.  The nature of a vertical approach means that the programming language itself must feature enough power and flexibility to support the implementation of server applications while also supporting strong sandboxing controls to allow user-defined code to be safely embedded and executed to extend and customize the behavior of the application at runtime.

Specifically, this is implemented by the Program class in Qore.  Sandboxing controls are implemented through Parse Options, which are combination of restriction, capability, parsing and/or execution rules that define the sandboxing rules for the logic container.

Custom APIs can be imported into Program objects through methods such as Program::importClass() and Program::importFunction().  Access to system functionality in sandboxed logic containers can be restricted by functional domain through Parse Options, or the Program object can be created with no Qore system API (i.e. standard functions, classes, and/or constants are not available), and then the desired API can be imported manually.


Using sandboxed Program objects in Qore programs allows for applications to be written in Qore that support runtime extensions also written in Qore.  The fact that these logic containers can be created and destroyed at runtime also allows for such applications to support runtime changes and logic upgrades without the need to restart the system for the changes to take effect.

Programmer productivity in such an application is maximized through the introduction of simple APIs for use in embedded code executed in specific application contexts, which, if properly implemented, allow for the behavior of an application to be customized in ways that could not be imagined when the application was designed and created, and therefore also allow for maximum flexibility.

Finding the right balance between configuration and coding to maximize productivity is important; allowing too much freedom (for example, if the balance of coding vs configuration is shifted too far towards coding) can increase programming complexity and steepen the learning curve, while allowing for too little customization through embedded code might cripple the application (particularly if the application is not very configurable through standard configuration).

In this way, such an application can be considered a special purpose meta programming language; programmers use a combination of configuration and embedded code to develop their solutions in the application.