What Is Apatar Open Source Data Integration?

by Alex KhizhniakApril 27, 2013
Learn about a great open-source ETL project we supported from 2006 to 2011.

Overview

The Gartner Group estimates that corporate developers spend 65% of their effort building bridges between applications. Altoros understands users’ need to complete their integration initiatives quickly and easily without the involvement of costly programming and coding efforts. That’s why in 2007 we delivered Apatar, an open-source data integration tool designed to help both business users and developers move their data in and out of numerous sources and formats.

No programming was required to accomplish even a complex integration when using Apatar—one needed to just “drag-and-drop” data between databases and apps using a visual mapping interface. Users could also benefit from reusing mapping schemas built by other community members.

The tool provided connectivity to Oracle, MS SQL, MySQL, Sybase, DB2, MS Access, PostgreSQL, XML, CSV, MS Excel, Compiere ERP, Salesforce.com, SugarCRM, Goldmine, InstantDB, any JDBC data sources, and many more. Apatar was 100% Java-based and platform-independent, running on Windows, Linux, and MacOS. The product was developed using Eclipse.

 

Popular scenarios

A robust, scalable Apatar architecture enabled end-users and developers to successfully solve a broad spectrum of integration needs—from extracting, transforming, and loading (ETL) for data warehousing, to migration, synchronization, and app integration.

Apatar could be used in diverse data-driven systems where you have information that needs to be captured and transferred to a new app, data warehouse, or a single front end. Popular integration scenarios:

  • Move data from one location to another
  • Synchronize data between two or more data sources
  • Integrate data across an enterprise
  • Consolidate data from two or more data sources into a data warehouse
  • Populate a data warehouse or a data mart
  • Validate and correct key data
  • Schedule backups

Apatar had a user-friendly UI and, even for a non-technical user, it would take just a couple of hours to get trained. Most users found it intuitive. Apatar had a number of built-in data quality tools, enabling data cleansing, de-duplication, etc.

We also promoted the concept of reusable DataMaps. Working with data sources, the tool generates XML metadata files (“DataMaps”) that store all the information captured by the Apatar designer and mapping graphical UIs. Apatar’s engine then manages, executes, and controls the total project execution plan described in the XML metadata files. In addition, for each executed job, Apatar generates a detailed report / log.

By adopting and following this metadata-centric approach, users can share the XML metadata files to reuse connectivity settings, transformation, and mapping. These XML metadata files can be sent to another user having the Apatar desktop app or to feed pre-built DataMaps to the Apatar server engine, saving time and money while integration.

 

Community and collaboration

Though Altoros was a major contributor, the project had an active community behind it, helping to improve the product and develop new connectors, as well as other functionality. The open-source code of Apatar allowed for building add-on connectors for any data integration applications, be it legacy, APIs, or semi-structured data sources / targets.

The Apatar manifesto stated:

  • Source Code. We believe that the quality of the core code base determines how easy or hard it is for all of us to work together and push Apatar forward.
  • Responsibility. We feel responsible for the success of the project, including the support of new contributors and facilitating communication across e-mail, mailing lists, and forums.
  • Presence. We strive—and encourage those who commit to the project—to be present in the mailing list and chat room so as to help fellow developers and new contributors feel less alone.
  • Helpfulness. Often, there just isn’t enough documentation available yet for new features and connectors. Sometimes there are questions to be answered. It is our goal to address every question in order to quickly solve the bottleneck standing between you and successful usage of Apatar.
  • Openness. We share our short and long-term goals with you in order to leverage all benefits of a joint community effort to build a truly open, great piece of software.
  • Founders commitment. We strive to be dedicated to work on the project, investing time for the further development of the software and the community.
  • Patience. We understand that every community is different and lives by its own rules. We believe that all of us should be patient with each other and support new members so that they can become vibrant community leaders.

 

Licensing model

Apatar used a “dual-licensing” business model. Under this model, users could choose to use the product either under the free software/open-source GNU General Public License (known as the GPL) or under a commercial license. Apatar’s dual-licensing was focused on promoting freedom in two ways: first, it encouraged the growth of free software by licensing Apatar under the GPL; second, it made it possible to use our software in situations where the GPL was not applicable.

Apatar’s dual-license model supported the company’s mission: to make data integration affordable to all, including those users who traditionally could not afford expensive proprietary data integration tools. The dual-licensing also created an opportunity for Apatar to give back to the community. The Apatar team believed that the open-source model of development and distribution is the most efficient way to produce high-quality software.

Renat Khasanshyn

“Open source is driven by real people like you and me, helping each other make our day-to-day work easier and more productive. Every contribution to the Apatar project extends the way companies manage their information, perhaps, one of the most valuable assets in the global economy. We value and appreciate each and every contribution you are making to the project.”

Renat Khasanshyn, Founder, Apatar

The dual-licensing business model was becoming increasingly popular among open-source companies at that time, paving the way for long-term financial viability. Other companies that offered dual-licensing included Alfresco, MySQL, SugarCRM, Trolltech, Zimbra, and others.

 

Afterword

By 2012, the tool was mostly deprecated, only the latest stable release (v1.12.*) was supported for some time. It still can be downloaded from the SourceForge. We do grateful to everyone who made this awesome project thrive. Check out these video tutorials or read exhaustive media coverage to learn more about the product.

Some of the most active Apatar contributors outside the core team

 

Further reading

 


Updated in April 2020.

Performance of Deep Learning Frameworks
  •  
  •  
  •