Building a Scalable Online Banking Environment at the Woolwich
By David Ellington (Issue 2 2001)
The Woolwich, a leading provider of financial services in the UK, created a 24x7x365 environment for its banking customers using Intel architecture, Dell servers, Microsoft's Distributed Component Object Model, the latest technologies for accessing the Internet, plus mobile phones and interactive digital TV. Their transition to Web-based services and e-business provides insight to other companies considering the move to e-commerce.
The Internet has forced businesses—from small companies to large-scale enterprises—to reconsider the way they operate. Establishing Web-based services and e-business, even for technologically savvy companies, can be daunting. But those businesses that make the transition can realize a huge payoff.
The Woolwich, a leading UK provider of personal financial services, offers savings and banking accounts, mortgages, loans, and general and life insurance. The bank wanted to include a new banking service focusing on customers as individuals. To do this, it needed to provide customers with 24x7x365 access to their accounts through a new range of access channels: the Web, mobile phones, and digital TV. The bank also wanted a scalable solution that could provide simultaneous Internet access for thousands of customers using this new technology.
Not only would this type of service benefit customers, but also the bank could reduce its internal processing costs and significantly improve both its products and the customer experience.
Customizing the banking system
The Woolwich chose the Intel® server environment to create a customized banking solution that would be available through PCs, mobile phones, digital TV, a call center, and an internal intranet. This server-based technology would allow seamless communication with the Woolwich mainframe, which held the organization's legacy systems. The initial hardware costs were lower, a wide variety of middleware was available for the Intel platform, and the system could be easily expanded if necessary.
Figure 1 shows the system design for a multi-access solution for the Woolwich. It incorporates the various options available to customers.
Figure 1. The Woolwich multi-access solution
The Woolwich had several critical requirements for its system architecture since the bank provides financial services to many customers.
Security is critical
Since Woolwich is a financial institution dealing with client money, security and reliability are critical factors. There can be no unexplained period of time offline. Availability must be 24x7x365, regardless of any technical problems that may occur.
Load balancing handles activity growth and surges
The Woolwich needed the ability to cope with ongoing increases in customer numbers. The business model was based upon "piloting" innovative financial products with a relatively small customer base, and when these proved successful, opening them up to the mass market. This meant that the system would have to scale radically from supporting a customer base of several thousand in the initial stages to one of several million in the longer term. An early architectural principle was established whereby it was determined that the most suitable mechanism for achieving the desired scalability would be to implement load balancing between each tier in the architecture.
This philosophy meant that it was unlikely that limitations in the scalability of the individual server platform would limit growth by becoming bottlenecks. It also meant that the server platforms could be based upon "commodity" servers rather than expensive proprietary hardware.
Some software available at the time included load-balancing capabilities; some did not. Load balancing for TCP/IP, for example, was available, but load balancing for Microsoft's Distributed Component Object Model (DCOM) was not. Yet the Woolwich needed software to allow for a random surge of processor activity generated by customers using Web browsers and an equal level of activity from call center operators at another time. The systems needed to deal smoothly with surges of activity from all the different channels deployed by the Woolwich.
Component-based architecture avoids rewriting code
Load balancing across different server tiers depends on a second critical technological decision: multiple access from different customer channels could only provide consistent, accurate results if the mid-tier server software was written as discrete, reusable, and independent components.
Business tasks were designed to be written just once and used by any channel. For example, a simple task or component such as "Get Account Balance" would check an account balance held on the mainframe, then deliver the answer to a mobile telephone, the Microsoft® Money channel, or a call center operator.
Other business tasks, such as password checking, error logging, auditing, and transferring funds between accounts were written as components. This design enables the Woolwich to add other tasks to meet new user requirements without rewriting code for the entire system, or to add new access channels as they are developed. The only rewriting required was the presentation layer functionality specific to TV, mobile phones, or the Internet.
Another major advantage to implementing component-based middleware is that the nature of the interface to the components is such that they can be modified to provide additional functionality without adversely affecting existing systems that use those components. For example, the "Get Account Balance" mentioned earlier could be expanded to provide the balance in another currency as well as the base currency. All existing functions would continue to work unchanged using the base currency balance, but new systems could display the alternate currency.
DCOM layer eases communication
Components can work only if a reliable interpretative layer exists between the varied access channels and the mainframe. Microsoft's DCOM model was crucial, because it was a trusted system that would ease communications between user channels and the Woolwich mainframe.
Elements of the implementation process
Most software worked as expected, but different combinations of software did not always provide the required outcome. Some software that initially performed well became too complex to manage or did not live up to expectations, so other software and hardware alternatives were chosen.
Although the original plan for the architecture proved correct, the Woolwich changed direction several times, generally in order to optimize the solutions or to replace those software packages that did not measure up to expectations. The Woolwich IT staff and contractors developed most of the component work, but AIT (www.ait.co.uk) wrote low-level code for the DCOM load balancer.
The Woolwich addressed elements of the implementation process with its goals of customer self-service and smooth operations. The Woolwich decisions and results concerning each element follow.
Performance
The Woolwich recently went live with its mobile telephone and interactive digital TV deployments. Although most of the component calls on the system arise from the call center, the Woolwich plan is to encourage as many customers as possible to use self-service through the many channels now available to them.
The site now processes more than 100,000 business object calls per hour and, on average, deals with 300 concurrent users. To date, the system can handle the number of Secure Sockets Layer (SSL) transactions that occur. If the number of users exceeds the system capacity, the Woolwich will simply install more servers to handle the extra load as needed. These may be Web servers, proxy servers, middleware component servers, or one of the other infrastructure servers.
Reliability
The Woolwich has reduced the need for complex programming and also simplified data recovery by developing software components using Visual Basic® and JavaTM , and choosing standard software and Intel-based hardware.
Availability
The system has no single point of failure. Two identical data centers provide continual service with a measured scheduled uptime in excess of 99.5 percent. If one data center goes down, the other takes over. Scheduled downtime is for a 30-minute window in the very early morning (UK time), one day per week.
The Woolwich also implemented automated testing tools ("smokers") to test the whole system, a specific part of the system, or a specific object on a specific server. These tools report when part of the system is not functioning correctly, even if the overall system is still working.
Scalability
By using loosely coupled clusters, the system can be scaled up as the number of users increases. The Woolwich rejected tightly coupled clusters-the alternative to loosely coupled clusters-because of complexity and cost. Effective load balancing was an essential part of the loosely coupled cluster strategy, as was the application architecture. All applications (and by implication, the components) are built to be stateless, meaning that each time they are invoked, they can be invoked on any server whereupon they will establish their "state."
Disaster recovery
The Woolwich implemented two distinct, identical data centers to provide disaster recovery. If one data center goes down, the other one takes over. In the near future, both data centers will be permanently operational with all traffic load balanced across both centers. This will provide both disaster recovery and normal operational capacity, avoiding the need for a "lights out" disaster recovery capability.
The data center architecture
A component-based architecture enables potential new channels to be easily deployed because the business application logic is centralized in the middle- and third-tier server levels. The architecture is designed to be resilient and reliable, with no single point of failure. By adding more servers to the respective clusters, more and more users can access the system. Costs remain low since the implementation uses commodity hardware and software. In addition, development is simple since Java and Visual Basic are easy to use and produce robust programs.
Figure 2 shows the data center architecture for the Woolwich system.
Figure 2. The data center architecture
Security
The Web bank and Microsoft Money use standard 128-bit SSL encryption. The Web bank provides further security through the encryption of all data and programs at the application level. This mechanism uses certificates provided by a trusted authority. Verisign® certificates are implemented on the two Web servers and also on the secure Web servers to provide secure communication in the Open Financial eXchangeTM (OFX) and the browser environment.
A firewall subsystem that uses a variety of techniques protects the complete installation.
Front-end architecture provides customer access
The front-end architecture for the Woolwich consists of many channels as described below. These access channels provide the means for customers to view their accounts and complete a variety of financial transactions.
The Web bank
The Web bank is a 100% Pure JavaTM -based application. Users access it through an Internet connection using a regular Java-enabled browser. The application accesses the Web servers by passing through the proxy. The Web servers include regular Web servers to access the Woolwich public, non-secure site and secure Web servers dedicated to all secure applications including the Web bank application.
Two Alteon® routers perform load balancing by dispatching the load (IP load balancing). These routers that comprise the cluster consist of a primary and a secondary system. Under normal circumstances, the primary system performs load balancing, while the secondary system monitors the health and performance of the primary system. If a problem occurs with the primary system, the secondary system automatically takes over (fails over) the load of the primary system.
The Java bank (Entranet) application running on top of the Internet Information Services (IIS) manages the user interface and accesses the business objects on the middleware servers using an XML data stream encapsulated in DCOM. The Java application also handles the encryption and decryption of the data sent over the Internet on top of an SSL connection.
The Microsoft Money channel
Users can access the service through a Microsoft Money application or Open Financial eXchange (OFX) by dialing in via the Internet. Through the proxy, this application accesses the Microsoft Financial Services Toolkit (MIFST) server. This server translates the request sent by Money in its own description language (an XML representation) into DCOM calls (again, encapsulating an XML data stream) to business objects resident on the cluster of middleware servers.
The MIFST servers are mirror images of one another. They run Microsoft IIS, the MIFST software, and application functionality developed by AIT to support Microsoft Money. Windows NT® Load Balancing Service (WLBS) handles load balancing of the requests on the MIFST servers. The MIFST crash recovery SQL Server provides services for the MIFST translation servers.
The WAP mobile phone
Users can access the site via mobile phone; that is, they access the NokiaTM wireless application protocol (WAP) server that sends the wireless markup language (WML) pages. This WAP server exchanges the data with the Web servers running WAP Active ServerTM Page (ASP) scripts that parse the WML. The ASP scripts run within IIS. As in other Web applications, these ASP scripts send their data and processing requests to the middleware servers using the XML data streams.
Interactive digital TV
When users connect to the OpenTV center using interactive digital TV, the data appears in a proprietary description language. The Web servers translate this data into XML format and, as for the other Web applications, send it directly to the middleware servers. Figure 3 shows this software architecture.
Figure 3. The software architecture
The call center
The call center accesses the middleware servers directly from the TSS3 application. TSS3, a call center application from AIT, is based on a "traditional" Windows NT fat client architecture. Each operator workstation runs its own instance of the AIT call center software and manages the windowing environment for the call center operator. When TSS3 needs to perform a transaction or needs access to core system data or system services such as logging in and authentication, then it will simply call the business objects resident on the middleware servers using the same XML stream encapsulated in DCOM that is used by the other channels.
The intranet
A Web server running IIS accesses the intranet. The branch locations of the Woolwich use two applications:
- GAP (Group Application Process): An intelligent application form-filling system. The GAP client is HTML with JavaScriptTM capable of running in Netscape Navigator® and Microsoft Internet Explorer. The GAP server-side uses objects written in C++ and Visual Basic.
- W.ISA (Woolwich Independent Savings Account): An application used to open a savings account associated with a Unit Trust (Note: An ISA is a British-specific tax-efficient investment product). The W.ISA client is an HTML and JavaScript application, but its server-side uses a third-party system called Amazon Integrator from Intelligent Environments.
|
In both cases, the server-side processes access the same business objects used by the previously described channels using the same DCOM encapsulated XML techniques.
The middle tier balances the load
The middleware servers provide business objects functionality to the front-end applications. These six servers, functionally identical, are dual homed; that is, they are connected to both the Ethernet and the Token-Ring environments (the new multi-channel infrastructure is Ethernet-based, while Token Ring is still used in the Woolwich legacy LAN environment). They run Microsoft Transaction Server (MTS) and a variety of other server packages, such as Microsoft SNA (System Network Architecture) server and Software AG's EntireX (used for mainframe communications). Servers have been added in the past and more servers will be added in the future. Most of the business components are written in Visual Basic, although a small subset is written in C++, but only where the use of C++ was technically unavoidable.
XML, the standard format used for all the internal data exchange streams, performs all data exchange between the front-end and the middleware components. Effectively, XML is used whenever a call is likely to pass across the network to another computer. Calls between processes or objects on the same computer will generally use a native message, such as DCOM, without the overhead of encapsulating within XML. The DCOM broker instantiates the different COM components on the middleware servers, passes the address of the component back to the calling server, then takes no further part in the interaction between the calling server and the component. This is the key to its efficiency.
The DCOM broker also does the load balancing for the middleware zone. It runs on a cluster of two Dell® servers managed by the Microsoft Cluster Server (MSCS). This cluster also runs SQL Server and Microsoft Message Queue Server (MSMQ), and provides transaction logging, customer authentication, and data recovery.
The cluster is an active/active cluster, but the individual services provided by the cluster are configured in an active/passive mode. The DCOM broker is active while SQL is passive on one server. On the other server, SQL is active and the DCOM broker is passive. If any given application fails on its active server, its functions are passed to the other server. The disk pods are configured as RAID-10. Each server has half of each pod allocated to it and three fully mirrored volumes:
- Internal disk drives for Windows NT and the cluster software
- One logical volume consisting of the database
- One logical volume for the database logs
|
One server acts as the primary domain controller, and another acts as the backup domain controller running the SQL Server databases.
The back-end contains the data
The middleware servers communicate with the Woolwich IBM® mainframe using the data link control (DLC) protocol encapsulating SNA LU6.2. The Software AG EntireX software manages this communication process with mainframe programs. It allows the business objects to access COBOL programs without the need to understand mainframe technology. Although it can potentially be used to allow the mainframe programs to be accessed as though they were DCOM components, this functionality is not currently being exploited. A Token-Ring network serves as the communications vehicle. The SNA link will, in the near future, be replaced with a direct TCP/IP link between the middleware servers and the mainframe.
The future: migrating to Windows 2000
Windows 2000 migration is a major component of the Woolwich enhancement plans for this environment. The advantages associated with Windows 2000 have already been proven in pilot projects. Significantly enhanced reliability and greatly improved manageability are among the attributes of Windows 2000 that we consider essential.
One negative aspect of adopting the "scale-out" approach is the complexity associated with managing large farms of servers. Future plans are to simplify this management by adopting a more rigorous approach to server builds through more extensive use of Windows scripting host (WSH) and SMS, and a more automated approach to application management using Microsoft Application Center Server. The Woolwich believes that Windows 2000 will significantly improve its ability to implement and support these and other tools.
Implementation of Windows 2000 is already underway: The appliance-type servers (for example, proxy servers) are the first candidates for conversion. Another benefit of the multi-server, loosely coupled cluster approach is that it is generally possible to selectively upgrade individual servers in a cluster, thus minimizing both the potential impact of change (if it should prove to be problematic) while simultaneously minimizing the need for a service outage as might otherwise be required in order to upgrade an entire cluster.
The conversion to Windows 2000 is expected to be largely completed by the second quarter of 2002.
The Woolwich offers true customer-focused banking
Using the Intel architecture in a distributed environment, the Woolwich reached its objectives of providing customers access to their accounts—24x7x365—using the customer media of choice, including new access devices like mobile phones or digital TV. Customers can now access their accounts and perform operations remotely using their favorite communications device.
The modularity of the architecture enables the Woolwich to easily adapt its business logic without touching the legacy mainframe used in the back-end environment. This ensures that the system will evolve over time. Using commodity hardware based on Intel hardware enables the Woolwich to scale the system easily with reasonable cost. Just by adding some new off-the-shelf systems, it can quickly increase the capacity of the environment.
We believe that the chosen architecture provides the greatest possible flexibility in terms of our future direction. Currently we "scale out," adding servers as required. But as new server hardware and more advanced processors (such as Intel's ItaniumTM ) enter the commodity marketplace, we will equally be able to "scale up" and the decision of whether to scale out or scale up can be based on the economic model most suitable when more capacity is required.
The banking business is mission-critical. Deploying a multi-tier architecture with no single point of failure and load-balancing techniques, the Woolwich delivers a reliable and scalable service to its customers.
David Ellington (david.ellington@woolwich.co.uk) is a systems architect responsible for the corporate-wide technical architecture at the Woolwich plc. Dave designed the technical infrastructure for the Woolwich multi-channel e-commerce systems and was a major influence in the philosophies and design principles used in the development of the Woolwich component-based middleware. He has worked in IT in South Africa, Holland, and the UK for a variety of employers in both the financial and manufacturing industries prior to joining the Woolwich.
For more information
For more information, technical support, and service, visit:
- www.intel.com/eBusiness
- www.ait.co.uk
- www.woolwich.co.uk
- www.microsoft.com
|
