Pre-release of STLport 5.1.5 for Symbian OS available for testing

The new version of STLport for Symbian OS is here! Actually this is just a pre-release that I'm making available for testing, and it's only distributed as source code. If you don't feel like compiling STLport by yourself (even though it's really simple and painless), then you'll have to wait for the official release.
I want to thank everyone who has helped me put together this pre-release by reporting anomalies, fixing bugs or simply leaving comments on this blog. But I still need your help! Please download this source package and test it against your existing code, and report any problems you may have with it.

Download
Get the source package at http://www.webalice.it/marco.jez/files/STLport-5.1.5-symbian-r1-RC.zip.

Platform compatibility
STLport can be used with all non-beta S60 and UIQ SDKs available at time of writing. Beta SDKs may work but I haven't tested them. Please note that I still have to test the library on non-Nokia devices.

Important notes

• The prerequisites have changed. Please read the INSTALL.txt file located in the ./symbian subdirectory. In a nutshell, this release requires PIPS 1.2 (or Open C MR) and Carbide.c++ 1.3. Older versions may or may not work, but they aren't officially supported. Previous versions of PIPS/Open C have bugs that also affect STLport, so you better stay away from them.
• STLport headers are now automatically installed into your SDK's epoc32/include directory. In order to use STLport from your applications, the directory to be added to the header search path is now /epoc32/include/stlport. The library name remains stlport_s.lib as in previous release.
• There is an experimental Python script that automatically builds all configurations of STLport for every SDK that is installed on your system. See INSTALL.txt for details.

Changes

• updated to STLport 5.1.5
• added support for UIQ 3.1 and S60 3rd Edition FP2
• fixed bug in _symbian_ecvt() which affected rendering of numerical values (thanks Peter Barth)
• fixed bug that affected the reference counting mechanism of the RFs instance in _fstream.cpp (thanks Peter Barth)
• improved project files and build scripts for easier installation
• forced null-pointer check after every allocation through operator new, because Symbian's operator new never throws std::bad_alloc
• minor cleanup to remove some annoying warning messages
• RTTI (std::type_info) is supported

Feedback is greatly appreciated!

New release of STLport for Symbian OS coming soon

Yes, I'm still alive. Thanks for asking. :-)

I apologize for the long delay since my last post, but I got captured by things more important than coding for fun... In the last few weeks, however, I found the time to update STLport for Symbian OS to the latest release of STLport, that is 5.1.5.

There are still some issues that need to be resolved, but I've already done most of the work. Before making the release and building the binaries for all supported SDKs, I'll make the source code available for testing. Although I've taken into consideration all comments and emails I've received so far, I couldn't reproduce all errors that were reported by users, so I'll need your help to test the new release!

What's new in this release? The main change is the upgrade to STLport 5.1.5 (see STLport's changelog for details). Some bugs have been fixed, build scripts and project files have been improved to make the installation from source code easier. Also, I've added support for RTTI (std::type_info), although it's still experimental.

The prerequisites have changed. In particular, the PIPS library version 1.2 or newer is required (it can be found in the PIPS 1.2 SDK or in the Open C Plugin MR). Older versions may not work. Note that the S60 FP2 SDK already contains the PIPS library, so no additional packages need to be installed. The "reference" IDE that I use for testing the library is now Carbide.c++ 1.3 Express. Older versions of Carbide may not work.

Stay tuned!

What's next?

What's coming after the release of STLport for Symbian OS? Well, at least a second release will follow shortly, possibly including a full implementation of STL locales and a fix for the missing typeinfo header in the platform SDKs. I'm also considering the possibility of building STLport as a DLL, or as a set of DLLs if a single binary is too large. This will help reduce the size of application executables, saving precious space on the device's memory.

The STL is just my first step towards a better development platform on Symbian OS. The second step will be longer and probably more difficult: porting Boost libraries. If you don't know what the Boost project is, please have a look here: http://www.boost.org. I think every serious C++ programmer should know Boost, regardless of whether he/she decides to use it or not. It's a matter of fact, some developers love Boost and others just can't live with it. I personally love it, in my opinion it is the perfect complement to the STL and to the C++ language itself. The reason why some programmers don't like it is that it is radically different from other libraries, which are usually based on (and often abuse) "ordinary" OOP paradigms and idioms. In contrast, Boost libraries try to enforce those features that are unique to the C++ language, such as compile-time polymorphism and template metaprogramming, resulting in higher-performance elegant code that can be extremely reusable if designed well. The downside is that using Boost is usually more difficult than using other libraries, and that's especially true for developers who don't write C++ programs "the C++ way". If you don't know what I mean, I suggest you to read this excellent book by Herb Sutter and Andrei Alexandrescu: C++ Coding Standards, a must for every C++ programmer.

Of course, neither the STL nor Boost libraries can replace the Symbian API and its nonstandard set of paradigms, rules and guidelines, but they can serve as a base for building an interface layer to abstract away the intricacies of Symbian programming and finally make writing software for this platform easier, safer and faster.

These are my long-term plans. Maybe they're too ambitious, given my lack of spare time...

STLport for Symbian OS released!

STLport for Symbian OS has been updated! A pre-release of version 5.1.5 (source code only) is available here.

I'm happy to announce the first full-featured release of STLport for Symbian OS 9.x!

STLport is a high-quality implementation of the C++ Standard Template Library (STL), a library that is part of the ISO/IEC C++ Standard and that is not included natively in the Symbian OS platform. By supporting the STL, Symbian applications can benefit from a wide range of existing C++ code that can now be ported easily.

This version of STLport should work on all Symbian OS 9.x devices, both S60 and UIQ variants. The library has been succesfully compiled with (and binaries are provided for) following platform SDKs:

• S60 3rd Edition 1.1 (Maintenance Release)
• S60 3rd Edition Feature Pack 1
• UIQ 3.0

The only dependency is a conforming Standard C library, that is provided by either the P.I.P.S. library (available for both S60 and UIQ devices) or Nokia's Open C plugin (S60 only).

All features of STLport have been ported, although locales other than the default "C" locale are not yet implemented. This will be done in next releases. All unit tests have passed on both the emulator and a wide range of devices, while only one test in the exception handling testsuite is still failing. More details are available in documentation files that come with the downloadable package.

So, now the interesting part. If you are using one of the aforementioned SDKs and you don't need to change any of the STLport configuration options, then you can download the prebuilt package that includes header files and binaries for a specific SDK:

Download prebuilt package for S60 3rd Edition 1.1 Maintenance Release SDK

Download prebuilt package for S60 3rd Edition Feature Pack 1 SDK

Download prebuilt package for UIQ 3.0 SDK

Download the README file that contains installation instructions

Please note that binaries for S60 were compiled against the OpenC library, while those for UIQ were compiled against the P.I.P.S. library. If you want to compile the library for another SDK or if you want to change the build configuration, you need the full source code so that you can recompile STLport from scratch:

Download full source code

The source code package also contains the necessary files for building an installable SIS package containing the STLport test suite.

If you have any questions or suggestions, please let me know. I can be contacted on the Forum Nokia's discussion forum, my nickname is "polo78".

Enjoy!

STLport coming tomorrow for both S60 and UIQ!

The first release of STLport for Symbian OS is ready, and the big news is... It works on UIQ 3.0 too! The only practical difference is the standard C library, which is Symbian PIPS on UIQ and Nokia's OpenC plugin on S60. These two libraries are actually very similar and have some parts in common, but whereas OpenC includes a "stdio server" for displaying and/or redirecting standard streams, PIPS apparently doesn't have such feature. This means that C-style console applications based on PIPS can't display messages through functions like printf(). Initially, this limitation was an issue because I couldn't run the STLport unit test program to test the library on the UIQ emulator or on the device (I've bought an used M600i phone from eBay), but I solved it by writing a simple shell application that launches the STLport test programs and captures their output.

So, everything is in place. There is a good chance that I'll be able to make the release available for download by tomorrow night.

STLport and the problem of destructors not being called in Symbian OS

While I was away I did some additional testing on my STLport variant for Symbian OS and I noticed an annoying fact: everytime a GUI application terminates, even gracefully, an ALLOC panic is raised on the emulator indicating a possible memory leak (which is a rather common bug in programs). It turned out that STLport uses a sort of optimized allocator to manage memory allocation and deallocation for STL containers and other classes. Such allocator is a global object that pre-allocates memory on its own strategy, but it doesn't free all allocated blocks when the program ends (I think that's because the operating system is supposed to do the work). This is not a problem under normal conditions, but Symbian GUI apps usually try to detect memory leaks by marking the heap and tracking memory usage throughout the program execution, thus raising an ALLOC panic when the STL-based applications ends. Although this is benign and doesn't happen on the real device (memory leaks are only detected in debug builds), it makes the useful memory leak check completely useless, which is bad.

After a bit of investigation I discovered a configuration option in STLport that forces the optimized allocator to release all allocated memory on program termination. I tried the same test with that option turned on, but nothing changed. Apparently, the reason is much more serious than I initially thought: it seems that class destructor of global objects in Symbian applications is never called, even though it is supposed to be called (as per the C++ ISO/IEC Standard) when the program exits cleanly. In the case of STLport, this means that memory still owned by the optimized allocator is not released on exit. This is a serious malfunction of the Symbian application model in my opinion.

I asked other developers at Forum Nokia an explanation for this fact, with no luck. Some users confirmed that destructors are not being called, but neither a motivation nor a workaround emerged from the discussion. For now, I'm simply switching off the optimized allocator in STLport, falling back to the "direct" malloc/free allocator, which may be a bit less efficient but at least it doesn't leave allocated memory around.

Back to work now, more news soon.

Native Symbian fstream implementation for STLport is done

As I mentioned in my previous post, the first attempt of porting the fstream component of STLport to Symbian OS using plain stdio functions (provided by OpenC) didn't produce good results. Performance was far from being acceptable, and the library had serious stability issues.
I've now implemented the fstream core in terms of native Symbian OS calls (RFs, RFile and the like) and the overall performance has sensibly improved. I don't get panics or strange behaviours anymore. Of course I can't guarantee it's 100% safe, as dealing with STLport code is not an easy task, and I'm a novice Symbian developer. Anyway, my implementation has passed all STLport regression tests, so I'm pretty optimistic.

The good news is, now all 385 unit tests pass on both the emulator and the device!

Today I've spent a lot of time trying to compile a simple "hello world" program based on STLport from the Carbide.c++ 1.1 Express IDE, with no luck. Although the app compiled and linked just fine from the command line (ABLD), the Carbide build process resulted in compiler failures (presumably caused by incorrect order of include paths) and mysterious linker errors. Fortunately the new Carbide.c++ 1.2 is now available for download (if only I had discovered it yesterday!). I'm pleased to say that this new version is much more robust and reliable than the previous one. More important, my STL-based "hello world" application builds and runs perfectly with Carbide.c++ 1.2.

So, I'm now in the final stages: recompile everything, double-check everything, write some README.blah.blah files and make the release. This is going to happen next week, however, as I'm taking a short holiday on Labour day.

Update on STLport for Symbian OS

This is an update for those of you who are following the development of STLport for Symbian OS in this blog. As I announced in my previous post the library is almost complete, but I've found that I/O through fstream is pretty unstable, especially in the emulator. After fixing a small bug in STLport I got the stdio-based implementation of fstream work, although with poor performance, on my N80 phone and on a few other devices through Remote Device Access. However, performance on the emulator is so horrible that I couldn't make fstream unit tests complete in a reasonable amount of time. Also, the emulator occasionally crashes or terminates the application with a panic when certain combinations of read/write operations are done.

I think the I/O routines in OpenC are quite buggy (as seems to be the family of printf functions, see here) because even simple code can occasionally fail or even crash. In particular, I'm observing a behavior similar to that described in this post at Forum Nokia, but it's not easily reproducible as it doesn't happen always (and not always under the same conditions).
As a result of two days wasted trying to make buggy things work, I've decided to try a better way: provide a custom implementation of fstream based on native Symbian OS calls rather than on OpenC calls. This shouldn't be too hard, but if I see that it's going to take too long then I'll just keep things as they are and I'll defer the fix until the next release.

Current status of unit tests is pretty encouraging though: only 3 tests out of 385 are still failing! I'm confident that I'll be able to make a release before next week.

Porting the C++ STL to Symbian OS 9.1

Over the last three days I've been working on adapting STLport, an open-source implementation of the C++ Standard Template Library (http://stlport.sourceforge.net), to the Symbian OS. My goal is to port all features of the library, including those parts that were deliberately ignored in previous porting attempts due to limitations of older Symbian systems, such as iostream, fstream and locale.

The standard C library that comes with the new OpenC plugin for S60 is the perfect ground for building STLport, which doesn't require any dependencies other than a decent compiler (both WINSCW and GCCE work perfectly) and a standard-compliant C library. Please note that I'm developing with S60 SDKs only, I don't know whether other kits (such as UIQ) come with a suitable C library or whether OpenC would work on such SDKs. So, for now, the port will be limited to S60 3rd edition.

I'm happy to say that the porting process is close to completion. I managed to build almost everything of STLport on both WINSCW and GCCE platforms and I'm now running unit tests and fixing things as I get errors. I think I've even found a bug in STLport that prevents fstream from working correctly (details here: http://sourceforge.net/tracker/index.php?func=detail&aid=1704843&group_id=146814&atid=766244). The only big part still missing is the support for locales, as it requires a customized implementation. Once I get the library up & running I'll dedicate some time to implement it, in the meanwhile a "dummy" locale implementation is present in order to avoid breaking existing code.

So, what are the motivations behind my decision to port the STL to Symbian? Well, the most obvious one is portability. I've written a lot of C++ applications in the past, especially non-GUI engines and OpenGL-based 3D apps, and all of them are based on the STL. Porting them to a new system/platform can be either a breeze, if the new development environment supports the STL, or a pain, if it doesn't. Moreover, sharing a single code base between different platforms can cut maintenance costs down to about half, which is good.

The second reason is that programming for Symbian is difficult. Antony Pranata once wrote an interesting post on this topic, see http://mobile.antonypranata.com/2005/12/19/why-is-symbian-os-development-very-difficult/. The STL, on the contrary, uses widely-recognized programming paradigms and, even better, it enforces good paradigms by exploiting the C++ language itself. Symbian libraries don't enforce the programming style they require, and I think this is the main reason (beyond the horrible, IMHO, coding style) that keeps novice Symbian developers away from the field. Such programmers are given too much responsibility (i.e. manage the cleanup stack manually) and they need to take care of too many aspects that don't have anything to do with the useful code they want to write. When they try and fail too many times, there is a good chance they will give up and never get back to Symbian again unless they have to.

I'm not going to start a flame about the quality of Symbian code. I'm pretty sure there are many experienced developers out there that don't find it difficult to write Symbian programs, but I also think that it's perhaps even more important to "open the doors" to new developers by making things easier and more standard-compliant. I do recognize the reasons behind Symbian architects' choices: when the OS was designed, the C++ language wasn't even standardized and lacked several features like exception handling, so they had to invent their own mechanisms (leave/trap, cleanup and so on). But come on guys, that was 20 years ago...

Anyway, back on topic. I think I'll be able to publish the final port of STLport for S60 3rd edition in few days. I'm going to release both source code and binaries for WINSCW (the emulator) and GCCE (the real phone). Using STLport will be of course possible from any development environment, including command-line (ABLD) and the Carbide.c++ family of products. Initially I'll compile STLport in the form of a static library, because the resulting binary is very large (8 MB) and making it a DLL would be overkill.

More news soon!