Loxcia by Ntico

Introduction

Ntico, a French consulting and software firm, specializes in data-driven applications and complex project management. One of their offerings is Locxia, a data platform aimed at providing businesses and local authorities with comprehensive insights into territories and their inhabitants. When faced with the challenge of managing enormous volumes of geospatial data, Ntico turned to Kestra as its orchestration platform.

Technology Stack

Ntico's technology stack is a blend of specialized tools, each serving a unique function but integrated to work in harmony:

• AWS S3: The primary data lake where both raw and processed data are stored.
• AWS Athena: Used for ad-hoc querying and data analysis.
• PostgreSQL: Manages transactional and relational data.
• Kestra: The orchestrator that ensures data flow between these components, handling workflow dependencies, error management, and scheduling.

Tackling Geospatial Data Challenges

Developing Locxia brought Ntico face-to-face with significant challenges: managing terabytes of geospatial data from a variety of sources, including APIs, open-data platforms, and raw satellite images, all while ensuring the scalability and cost-effectiveness of the solution. The diversity and volume of the data sources, coupled with the need for a scalable and cost-effective solution, underscored the complexities of the project.

Choosing Kestra

Kestra stood out as the optimal solution for several reasons. Its prowess in handling advanced data ingestion and processing allowed Ntico to efficiently manage data from varied sources, channeling it into an AWS S3 bucket. The platform's user interface offered a clear visual representation of task dependencies, facilitating effective workflow management. Furthermore, Kestra's compatibility with Docker enabled Ntico to infuse the platform with their business logic and technical requirements, offering a degree of customization that was pivotal for the project's success.

Efficient Data Management and Scalability

The adoption of Kestra enabled Ntico to effectively address the challenges they faced with Locxia. It not only streamlined the data ingestion process, accommodating data from diverse sources, but also proved scalable, meeting the growing demands of the platform. The intuitive interface and the ability to tailor data pipelines according to specific needs empowered Ntico to design and implement solutions that were both robust and highly adaptable.

Conclusion

The collaboration between Ntico and Kestra has been critical in the development of Locxia. Kestra's advanced orchestration capabilities, combined with its scalability and customization options, have played a crucial role in enabling Ntico to deliver a powerful, reliable, and efficient data platform. This partnership highlights Kestra's contribution to driving data-driven innovations, allowing Ntico to offer rich territorial insights and demographic analyses.