High Level System Architecture
High Level Architecture
SCSB is designed to enable efficient, real-time transactions between the partner integrated library systems (ILS), their web-accessible discovery environments, and the inventory management system (IMS) or storage facility’s system.
One side SCSB talks to multiple institution library systems using SIP2 or NCIP or REST APIs and on the other side it talks to inventory management systems using REST APIs.
SCSB is designed to provide
Real-time availability of all shared collection items in OPAC systems
Validate and process the requests in real-time
Provide real-time item status
Integration with Partner ILS to support requesting and circulation.
Eliminate cross loading bibliographic and item records in ILS (Voyager, Sierra, Alma)
The following diagram depicts the the environment (of the collaborating systems) where the solution fits in. SCSB is the middleware solution.
Layered Architecture
The architecture includes five distinct layers:
Presentation Layer
Authentication Layer
Services Layer
Search and Data Services Layer
Data Layer
Presentation Layer
The presentation layer deals with user interface aspects of the system. SCSB’s user interface is developed using Angular JS.
Authentication Layer
SCSB supports both CAS (Central Authentication Service) and OAuth 2.0
Microservices Layer
The microservice layer encapsulates business rules, which are made available to the presentation layer. The presentation layer requests enterprise services, which are then fulfilled by this layer. The architecture envisages providing a seamless enterprise service layer communicating with internal data stores and 3rd party services. The data access layer supports the enterprise service layer by serving the data required.
SCSB has the following microservices.
Microservice | Functionality | Notes |
|---|---|---|
SCSB-Gateway | Gateway for All APIs | This is an internal gateway for all API calls |
SCSB-Circ | Request Processing | This microservice handles all circulation related aspects of the system, like processing requests in all the supported formats( SIP2, NCIP and REST API). This service also handles loading of the legacy circulation transactions. |
SCSB-Doc | Solr Index, Search | This microservice in the front-end interface to the solr server ( index data of bibliographic records). This service also handles Matching Algorithm and related reports |
SCSB-ETL | Initial Data Load | This microservice is primarily for ETL related aspects like data loads and data exports and related reports |
SCSB-Auth | User Authentication | This microservice is primarily for authentication, authorization and session management aspects of the system |
SCSB-UI-Angular | User interface for Search | This microservice is primarily for the user interface layer of various modules. |
SCSB-UI-Services | Services required for SCSB-UI-Angular | This microservice is primarily to provide the necessary data to the UI |
SCSB-Batch-Scheduler | Configuring Batch Jobs | This microservice is primarily for managing batch scheduling jobs. |
SCSB-Solr-Server | Apache Solr Server | This microservice is primarily to create, store and retrieve index of the meta data of the bibliographic records for all partners |
SCSB-ActiveMQ | Apache ActiveMQ server | This microservice is to handle all messaging requirements of SCSB |
SCSB-ZK | Apache Zoo Keeper | This microservice is to maintain the synchronization of solr-index data between the servers |
Data Services Layer
The data services layer provides fundamental services to fulfill the business needs (fulfilled through enterprise services) such as Search, Request Item, etc. The data services layer serves data required by enterprise services. Data services support both relational database and Solr.
Services implementing data access to relational database leverages Java Persistence Architecture (JPA), providing separation of object persistence and data access logic from a particular persistence mechanism (relational database) in data layer. This approach provides the flexibility to change the applications persistence mechanism without the need to re-engineer application logic that interacts with the data layer. Persistence classes are developed following the object-oriented idiom including association, inheritance, polymorphism, composition, and collections. This framework provides the flexibility to express queries in its own portable SQL extension, as well as in native SQL, or with object-oriented criteria.
Services implementing data access to Solr / Lucene search wraps the Solr RESTFul API's to provide features such as search, filter, sort and navigation.
Data Layer
The data layer serves as the data store for all persistent information in the system including the relational database and search engine indexes.
RDBMS data layer is comprise of MySQL server. RDBMS data layer is accessed only from the data access layer via Data Access Objects (DAOs). RDBMS architecture allows a single physical database to be accessed by concurrent instances running across several different CPUs. The data layer is composed of a group of independent servers or nodes that operate as a single system. These nodes have a single view of the distributed cache memory for the entire database system providing applications access to more horsepower when needed while allowing computing resources to be used for other applications when database resources are not as heavily required.
A single Solr instance can support more than one index using Solr cores (single index per core). A single large index can be a performance overhead. SolrCloud distributes a single index on different machines, commonly referred as shards. All shards of the same index making one large index are referred as collection. While collection supports index scaling, it does not provide redundancy. Replication of shards provides redundancy and fault tolerance.
Zookeeper maintains the SolrCloud, by distributing the index across shards and federating the search through the collection. SolrCloud uses leaders and an overseer. In the event of leader or the cluster overseer failure, automatic failover will choose new leaders or a new overseer transparently to the user and they will seamlessly takeover their respective jobs. Any Solr instance can be promoted to one of these roles.