Pre-release of STLport 5.1.5 for Symbian OS available for testing

The new version of STLport for Symbian OS is here! Actually this is just a pre-release that I'm making available for testing, and it's only distributed as source code. If you don't feel like compiling STLport by yourself (even though it's really simple and painless), then you'll have to wait for the official release.
I want to thank everyone who has helped me put together this pre-release by reporting anomalies, fixing bugs or simply leaving comments on this blog. But I still need your help! Please download this source package and test it against your existing code, and report any problems you may have with it.

Download
Get the source package at http://www.webalice.it/marco.jez/files/STLport-5.1.5-symbian-r1-RC.zip.

Platform compatibility
STLport can be used with all non-beta S60 and UIQ SDKs available at time of writing. Beta SDKs may work but I haven't tested them. Please note that I still have to test the library on non-Nokia devices.

Important notes

• The prerequisites have changed. Please read the INSTALL.txt file located in the ./symbian subdirectory. In a nutshell, this release requires PIPS 1.2 (or Open C MR) and Carbide.c++ 1.3. Older versions may or may not work, but they aren't officially supported. Previous versions of PIPS/Open C have bugs that also affect STLport, so you better stay away from them.
• STLport headers are now automatically installed into your SDK's epoc32/include directory. In order to use STLport from your applications, the directory to be added to the header search path is now /epoc32/include/stlport. The library name remains stlport_s.lib as in previous release.
• There is an experimental Python script that automatically builds all configurations of STLport for every SDK that is installed on your system. See INSTALL.txt for details.

Changes

• updated to STLport 5.1.5
• added support for UIQ 3.1 and S60 3rd Edition FP2
• fixed bug in _symbian_ecvt() which affected rendering of numerical values (thanks Peter Barth)
• fixed bug that affected the reference counting mechanism of the RFs instance in _fstream.cpp (thanks Peter Barth)
• improved project files and build scripts for easier installation
• forced null-pointer check after every allocation through operator new, because Symbian's operator new never throws std::bad_alloc
• minor cleanup to remove some annoying warning messages
• RTTI (std::type_info) is supported

Feedback is greatly appreciated!

New release of STLport for Symbian OS coming soon

Yes, I'm still alive. Thanks for asking. :-)

I apologize for the long delay since my last post, but I got captured by things more important than coding for fun... In the last few weeks, however, I found the time to update STLport for Symbian OS to the latest release of STLport, that is 5.1.5.

There are still some issues that need to be resolved, but I've already done most of the work. Before making the release and building the binaries for all supported SDKs, I'll make the source code available for testing. Although I've taken into consideration all comments and emails I've received so far, I couldn't reproduce all errors that were reported by users, so I'll need your help to test the new release!

What's new in this release? The main change is the upgrade to STLport 5.1.5 (see STLport's changelog for details). Some bugs have been fixed, build scripts and project files have been improved to make the installation from source code easier. Also, I've added support for RTTI (std::type_info), although it's still experimental.

The prerequisites have changed. In particular, the PIPS library version 1.2 or newer is required (it can be found in the PIPS 1.2 SDK or in the Open C Plugin MR). Older versions may not work. Note that the S60 FP2 SDK already contains the PIPS library, so no additional packages need to be installed. The "reference" IDE that I use for testing the library is now Carbide.c++ 1.3 Express. Older versions of Carbide may not work.

Stay tuned!

Import/export nightmare

First of all I apologize for the lack of writing in the last weeks. I wish I could find the time to write more regularly. I'm still working hard on STLport for Symbian OS, and I'm also building my own website where I plan to host development projects (included STLport), some of my work on OpenSceneGraph, a section dedicated to technical articles, and more. This blog will also move to the new home when it's ready.

One of the main requirements for the second release of STLport for Symbian OS is to provide a DLL build target in addition to the already provided static library target. The common reasons for building a library as a DLL are well known, but the main reason for doing this on Symbian comes from an unwanted "feature" of the build chain provided with Carbide.c++ Express: it seems that if you have a static library a.lib and another static library b.lib that depends on the former, trying to build an application that uses b.lib results in linker errors because symbols defined in a.lib that are used in b.lib are considered as undefined, even though they should be found because both libraries are being linked together. In other words, it seems that Carbide.c++ checks static libraries for undefined symbols once at a time without considering the "big picture" of all libraries being linked together. This of course makes the build fail because dependencies between static libraries always result in linker errors, hence the need for a DLL build target.

So, what's the difficulty in providing a DLL build target? Unfortunately, it's not just matter of modifying build options. One has to mark every function and variable declaration with the IMPORT_C macro, and their definition with the EXPORT_C macro. Usually, large projects like STLport already contain some sort of dllexport/dllimport macro system, but the way it works in Symbian is (once again) radically different from other systems, so you can't rely on it. In a nutshell, here is the difference:

Windows - MSVC

• The implementation (definition) is never flagged with import/export macros
• The interface (declaration) is flagged with a macro that assumes an "export" meaning when the library is being built, and an "import" meaning when the library is being used by client code
• When a class is flagged, all of its members are automatically imported/exported

Symbian - GCCE/WINSCW

• The implementation is always flagged with EXPORT_C
• The interface is always flagged with IMPORT_C
• All class members that need to be imported/exported must be flagged individually

So, even though the goals are similar if not the same, the approaches are different and not compatible. This means that a lot of manual work is potentially necessary to make STLport a Symbian DLL, especially if you are not very familiar with the code (and I'm not).

Given my commitment to port other libraries (like Boost) to Symbian, I've decided to invest some time to find a way to do the "dirty work" automatically by analyzing source files and identifying those declarations and definitions that need to be imported or exported. This is a tough task that is driving me nuts. I initially tried with Doxygen's XML representation, but I found it inconsistent when it comes to parse heavily templatized code like that of STLport. Then I tried GCC-XML but it seems to be using an old version of GCC that can't parse STLport's source files correctly.

I'm currently experimenting with a C++ front-end from EDG which is provided in the Program Database Toolkit package from the University of Oregon. Unfortunately, I can't use the intermediate language representation produced by the EDG front-end because it's in binary format and its documentation is not included in the PDT project (the front-end is a commercial product, and its internal documentation can't be redistributed). The front-end also generates a cross-reference file which is documented, and that's the file I'm currently experimenting with. It doesn't contain all the information I need on declarations and definitions, but I give it a try.

Development of STLport for Symbian is all but dead. I just hope to solve the above problems soon. I wish to thank all of you who have sent me feedbacks in the last weeks, I really appreciate it. Please keep testing STLport!

What's next?

What's coming after the release of STLport for Symbian OS? Well, at least a second release will follow shortly, possibly including a full implementation of STL locales and a fix for the missing typeinfo header in the platform SDKs. I'm also considering the possibility of building STLport as a DLL, or as a set of DLLs if a single binary is too large. This will help reduce the size of application executables, saving precious space on the device's memory.

The STL is just my first step towards a better development platform on Symbian OS. The second step will be longer and probably more difficult: porting Boost libraries. If you don't know what the Boost project is, please have a look here: http://www.boost.org. I think every serious C++ programmer should know Boost, regardless of whether he/she decides to use it or not. It's a matter of fact, some developers love Boost and others just can't live with it. I personally love it, in my opinion it is the perfect complement to the STL and to the C++ language itself. The reason why some programmers don't like it is that it is radically different from other libraries, which are usually based on (and often abuse) "ordinary" OOP paradigms and idioms. In contrast, Boost libraries try to enforce those features that are unique to the C++ language, such as compile-time polymorphism and template metaprogramming, resulting in higher-performance elegant code that can be extremely reusable if designed well. The downside is that using Boost is usually more difficult than using other libraries, and that's especially true for developers who don't write C++ programs "the C++ way". If you don't know what I mean, I suggest you to read this excellent book by Herb Sutter and Andrei Alexandrescu: C++ Coding Standards, a must for every C++ programmer.

Of course, neither the STL nor Boost libraries can replace the Symbian API and its nonstandard set of paradigms, rules and guidelines, but they can serve as a base for building an interface layer to abstract away the intricacies of Symbian programming and finally make writing software for this platform easier, safer and faster.

These are my long-term plans. Maybe they're too ambitious, given my lack of spare time...

STLport for Symbian OS released!

STLport for Symbian OS has been updated! A pre-release of version 5.1.5 (source code only) is available here.

I'm happy to announce the first full-featured release of STLport for Symbian OS 9.x!

STLport is a high-quality implementation of the C++ Standard Template Library (STL), a library that is part of the ISO/IEC C++ Standard and that is not included natively in the Symbian OS platform. By supporting the STL, Symbian applications can benefit from a wide range of existing C++ code that can now be ported easily.

This version of STLport should work on all Symbian OS 9.x devices, both S60 and UIQ variants. The library has been succesfully compiled with (and binaries are provided for) following platform SDKs:

• S60 3rd Edition 1.1 (Maintenance Release)
• S60 3rd Edition Feature Pack 1
• UIQ 3.0

The only dependency is a conforming Standard C library, that is provided by either the P.I.P.S. library (available for both S60 and UIQ devices) or Nokia's Open C plugin (S60 only).

All features of STLport have been ported, although locales other than the default "C" locale are not yet implemented. This will be done in next releases. All unit tests have passed on both the emulator and a wide range of devices, while only one test in the exception handling testsuite is still failing. More details are available in documentation files that come with the downloadable package.

So, now the interesting part. If you are using one of the aforementioned SDKs and you don't need to change any of the STLport configuration options, then you can download the prebuilt package that includes header files and binaries for a specific SDK:

Download prebuilt package for S60 3rd Edition 1.1 Maintenance Release SDK

Download prebuilt package for S60 3rd Edition Feature Pack 1 SDK

Download prebuilt package for UIQ 3.0 SDK

Download the README file that contains installation instructions

Please note that binaries for S60 were compiled against the OpenC library, while those for UIQ were compiled against the P.I.P.S. library. If you want to compile the library for another SDK or if you want to change the build configuration, you need the full source code so that you can recompile STLport from scratch:

Download full source code

The source code package also contains the necessary files for building an installable SIS package containing the STLport test suite.

If you have any questions or suggestions, please let me know. I can be contacted on the Forum Nokia's discussion forum, my nickname is "polo78".

Enjoy!

STLport coming tomorrow for both S60 and UIQ!

The first release of STLport for Symbian OS is ready, and the big news is... It works on UIQ 3.0 too! The only practical difference is the standard C library, which is Symbian PIPS on UIQ and Nokia's OpenC plugin on S60. These two libraries are actually very similar and have some parts in common, but whereas OpenC includes a "stdio server" for displaying and/or redirecting standard streams, PIPS apparently doesn't have such feature. This means that C-style console applications based on PIPS can't display messages through functions like printf(). Initially, this limitation was an issue because I couldn't run the STLport unit test program to test the library on the UIQ emulator or on the device (I've bought an used M600i phone from eBay), but I solved it by writing a simple shell application that launches the STLport test programs and captures their output.

So, everything is in place. There is a good chance that I'll be able to make the release available for download by tomorrow night.

STLport and the problem of destructors not being called in Symbian OS

While I was away I did some additional testing on my STLport variant for Symbian OS and I noticed an annoying fact: everytime a GUI application terminates, even gracefully, an ALLOC panic is raised on the emulator indicating a possible memory leak (which is a rather common bug in programs). It turned out that STLport uses a sort of optimized allocator to manage memory allocation and deallocation for STL containers and other classes. Such allocator is a global object that pre-allocates memory on its own strategy, but it doesn't free all allocated blocks when the program ends (I think that's because the operating system is supposed to do the work). This is not a problem under normal conditions, but Symbian GUI apps usually try to detect memory leaks by marking the heap and tracking memory usage throughout the program execution, thus raising an ALLOC panic when the STL-based applications ends. Although this is benign and doesn't happen on the real device (memory leaks are only detected in debug builds), it makes the useful memory leak check completely useless, which is bad.

After a bit of investigation I discovered a configuration option in STLport that forces the optimized allocator to release all allocated memory on program termination. I tried the same test with that option turned on, but nothing changed. Apparently, the reason is much more serious than I initially thought: it seems that class destructor of global objects in Symbian applications is never called, even though it is supposed to be called (as per the C++ ISO/IEC Standard) when the program exits cleanly. In the case of STLport, this means that memory still owned by the optimized allocator is not released on exit. This is a serious malfunction of the Symbian application model in my opinion.

I asked other developers at Forum Nokia an explanation for this fact, with no luck. Some users confirmed that destructors are not being called, but neither a motivation nor a workaround emerged from the discussion. For now, I'm simply switching off the optimized allocator in STLport, falling back to the "direct" malloc/free allocator, which may be a bit less efficient but at least it doesn't leave allocated memory around.

Back to work now, more news soon.