Here is another example for an application developed with the toolchain we developed over the time. It’s a CATI software system – get interviews by calling persons for different purposes.
The system is similiar to the election system I mentioned in spring this year on this site. The base component for this system is the oo-database Gemstone/S. Several external applications are communicating with the database via a HTTP-RPC API programming style.
The application itselfs is also pretty much different – its much more multitasking. Lots of concurrent users may connect to the database, background jobs are executing and so on.
The communication between clients and database is normally done via browser-based applications (written in Sencha ExtJS) and the database via HTTP. The database itself is not a frontend http-server, but all components are tunneled through the original HTTP-server: an Apache/2.
The requests are handled by several Topaz processes, with different memory contraints – all API calls are defined according to the “believed” memory consumption during requests execution. Load-Balancing is done by Apache/2 and it leads the requests to the Topaz process with the correct memory configuration.
The much more interesting part is the introduction of ZeroMQ. The database now sends “events” (indicating some domain events like login, logout, interview ended, ..) over a “publish-subscribe” socket pattern to external applications. Most of these service programs are written in Python 3.5.2.
But also the browser based application need some events and we wrote a Python based ZeroMQ <-> WebSocket relay. The relay is also tunneled through the Apache/2 server.
Now the browser application – when doing active work – may communicate with the database with HTTP, but are also awaiting informations via incoming events received via a WebSocket connection. In future we may only use WebSockets for all kind of application – we will see.
Some additional tools are written in Python scripts: interfaces to dialer systems, to time-management systems – and simulated interviewers 🙂
These simulated interviewer programs are the most interesting things – they show the concurrency problems within the whole structure. They also show, that 400 interviewers working against this application server are not a load problem. The problem will be the 1-5 supervisors doing lots of statistic data and to visualize the current state of a CATI studio. These few users produce much more CPU work !
The high concurrency of this system also pops up another interesting fact: simple jobs become difficult: transfering 100000 domain objects from one queue to another queue become now a candidate for conflicts. Such jobs have to be rewritten in a adapting way, so that the jobs can be done in multiple smaller jobs etc.
The longer the API calls needs, the more is the risc of conflicts and transactions are aborted.
And of course we also build fat clients from our www-clients by using the Election technology. They are working – but we still have to think about, how to improve such a fat client in contrast to a pure web-client.
Drawbacks found ? Well yes: a more structured query language would be nice.