Search-based application
Search-based applications (SBA) are software applications in which a search engine platform is used as the core infrastructure for information access and reporting. SBAs use semantic technologies to aggregate, normalize and classify unstructured, semi-structured and/or structured content across multiple repositories, and employ natural language technologies for accessing the aggregated information.
Pre-Conditions
SBAs are typically deployed when there is a need to synthesize heterogeneous content from multiple sources (text documents, multimedia files, database content, etc.), or, more commonly, they are used to replace traditional relational database systems as the primary data access infrastructure when one or more of the following constraints exist:
- High volumes of data and/or users:
Search engines are optimized for fast processing of access requests (read operations), while databases are optimized for recording and storing transactions (write operations). Accordingly, search engines can provide efficient processing against very large data sets by a high number of simultaneous users (whether human users or other software applications).
- A need for real time information:
To maintain the referential integrity of data and balance transactional and access requirements, large databases are typically updated in batch processes, resulting in data latency. Search engines maintain indexes using continual incremental and differential update processes, supporting real-time data availability.
- A need for ad hoc access and/or reporting against a broad range of criteria:
Search engines enable users to access and manipulate data according to any criteria maintained in the index— whether the criteria is extracted from a source system or created by the engine during the course of natural language processing—without advance programming of all queries and views.
- The need to extend access to specialist systems to untrained users:
Because SBAs can extract data from complex systems and make it available in an independent data layer that can be accessed using technologies such as natural language search, fuzzy matching and faceted navigation, SBAs are being used to extend access to the content in complex, specialist systems to non-specialists.
Practical Uses
SBAs are used for a variety of purposes, including:
- Enterprise Business Applications: For example, Customer Relationship Management (CRM), Enterprise Resource Planning (ERP), Supply Chain Management (SCM), Compliance & Discovery, and Business Intelligence (BI)
- Web Applications: Typically, B2B, B2C and C2B applications that mash-up data and functionality from diverse sources (databases, Web content, user-generated content, mapping data and functions, etc.)
- Database Offloading: In this case, SBAs are used to provide an alternate means of accessing database content that is non-intrusive to source systems
- Data Migration: SBAs are sometimes used as a temporary platform to ensure continuity of information access during large scale migration projects
- Information Lifecycle Management: SBAs are used as a complement to ILM processes and ecosystems, such as those used for Product Lifecycle Management (PLM) and Master Data Management (MDM)
The use of a search platform as the core infrastructure for software applications has been enabled largely by two evolutions in search engine technology: 1) the capability of later generation engines to retain and exploit the semantics embedded in structured data, and 2) the integration of mathematical and statistical processors to provide reporting, analysis, and, occasionally, geospatial capabilities.
Search engines are not a replacement for database systems; they are a complement. They have been optimally engineered to facilitate access to information, not to record and store transactions. In addition, the mathematical and statistical processors integrated to date into search engines remain relatively simple. At present, therefore, databases still provide a more effective structure for complex analytical functions.
References
- Butler Group Webinar on Search Based Applications explaining SBA and how they work
- Presentation on Search Based Applications by Information Builders
- IDC Executive Brief "The Information Advantage: Information Access in Tomorrow's Enterprise," October 2009, downloadable from the Exalead.com website. Adapted from Hidden Costs of Information Work: A Progress Report and Worldwide Search and Discovery Software 2009–2013 Forecast Update and 2008 Vendor Shares by Susan Feldman, IDC.
- IDC Search and Discovery Software: 2009 Market Map
- KMWorld article Search-based applications support critical decision making
- Kellblog post IDC's Definition of Search-Based Applications
- Steve-Kearns' Building Multilingual Search Based Applications presentation at Apache Lucene EuroCon 2010 conference
- Information Today article Attivio Upgrades Its Active Intelligence Engine