This business intelligence solution can obtain data from SQL databases, MS Excel files, tables, text files, as well as from social networks, including Salesforce, Facebook, and LinkedIn while preserving the exact structure and content of any document. Users can pivot rows and columns, as well as join, aggregate, and transform data. Furthermore, it’s possible to sort or filter information according to certain criteria. Processed data can be uploaded to any database or system selected as the output source. Users can create and map data flows. The solution features an interface typical of Microsoft products and the editing process is based on the WYSIWYG principle. It means that users can see how data will be transformed and add new relations between the rows and columns with a mouse click.
Science Warehouse brings cloud-based services and data analysis technologies to procurement organizations. The company offers a range of solutions that simplify purchasing processes by delivering savings and provide real-time desicion support based on collected analytical data.
The customer wanted to preserve the core architecture of the existing Java application and keep the current version of the Oracle database to avoid additional costs. Apart from buying a more expensive license to upgrade to the Oracle Database Enterprise Edition, the company would have to migrate data from the legacy database to a new store and spend time on testing, tuning, etc. Science Warehouse wanted to check out if there was a NoSQL solution that could help them to improve performance without huge investments like this.
While the agenda of the first day, as described above, mainly covered the theoretical aspects of the new non-relational databases, the second day was dedicated to practical examples and actual suggestions for optimization. Altoros’s big data experts pointed out some bottlenecks that were detected while reviewing the system and the database that had been carried out prior to the workshop. They provided some recommendations on how to improve the system and also demonstrated how this can be done.
Altoros suggested using the polyglot persistence approach for optimizing the Science Warehouse solution. It would enable the company to store different types of data in different databases: transactional data would be stored in the Oracle SQL database, while non-sensitive, quickly growing data (such as logging information, price history, etc.) would be kept in a NoSQL database.
During a two-day workshop, Science Warehouse’s engineers got enough detailed information on NoSQL to evaluate the possibilities and start prototyping themselves. The recommendations from Altoros helped the company to point in the right direction for the future of their system and define the areas they should focus on. The given practical examples continuously referred to their current setup and everyday work, which helped Science Warehouse to rapidly see the difference between SQL and NoSQL solutions and learn about the possibilities of a multi-database system. The company’s engineers analyzed Altoros’s recommendations and started prototyping the suggested features.
The workshop showed that it is not necessary to re-write a system from scratch or even refactor it to solve scalability and performance issues. The NoSQL database technologies can quite easily be introduced into existing systems to speed up operations, gain scalability, and avoid upgrading to expensive licenses.