Preparing for the 0.8.4 Release

I just finished doing a major rewrite of the internal lvalue handling in Qore.  Basically now most types lvalues are stored in a union which consists of one of a 64-bit int, a double, a bool, or a pointer to a generic Qore value object.

The thing with Qore is that at the beginning, all values were dynamically allocated objects derived from a common virtual base.  This was to allow for atomic reference counts and a copy-on-write approach to managing data, which is very efficient for large data structures.  In this way you can pass a large data structure (such as a hash, list, or object) to a function by value, and the value is only copied if it's changed.  Even then, only the top level of the data structure is copied, because each of the values is also a reference-counted object, so, unless they change as well, they are only copied by reference (meaning a pointer is copied and its reference count is incremented).

However this approach is not efficient for small, discrete values such as integers and floating-point values.  It's even worse for boolean values, which can be stored in as little as 1 bit.

Qore has an optimization for special values like True and False and some others, whereas there is only 1 single value in the Qore library that is not subject to reference counting.  However this is not possible with ints and floats.

The solution that I implemented for lvalues is to use the union as described above; the type of the union is set based on the lvalue's type restriction -- so if you declare a local variable as "int" or "softint", then it will be internally stored and operated on only as an integer (the same with "float" or "softfloat").

This allows Qore to store and operate directly on the base data type, instead of always working with another level of indirection (a pointer to a generic value object) and also eliminates the associated dynamic memory management.   So this approach has both memory and speed benefits.

This work showed me a clear way forward for doing some very cool optimizations in Qore regarding value handling - basically long-term I plan on making all Qore values some sort of union like this, which will allows Qore always in every instance to operate directly on the base data type when possible.  This will be necessary before starting llvm integration as well.

This will be a lot of work and will start some time after the upcoming 0.8.4 release.

I also implemented user thread initialization - you can now set a closure or call reference to be executed any time a new thread is started in the current Program object (or any time another Program object accesses the current Program object in a new thread) - this can be used to initialize thread-local data in the Program.

Also I implemented an optional maximum size for the Queue class - if a maximum size is set then writes to the Queue will block if the Queue already has the maximum allowed number of elements in it.  In this way, Qore Queue objects can be used like a buffered Go channel.

At the moment, Qore 0.8.4 is feature complete and stable in svn, however there's still some more packaging work etc to be done before the actual release, which hopefully will be pretty soon (I'm aiming for sometime in the next 2 weeks).